[GitHub] [spark] AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529987369 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu

[GitHub] [spark] AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25708: [SPARK-28141][SQL] Support special date values URL: https://github.com/apache/spark/pull/25708#issuecomment-529987356 Merged build finished. Test PASSed. This is an automated m

[GitHub] [spark] xianyinxin commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2

2019-09-10 Thread GitBox
xianyinxin commented on issue #25626: [SPARK-28892][SQL] Add UPDATE support for DataSource V2 URL: https://github.com/apache/spark/pull/25626#issuecomment-529989458 Thank you very much @rdblue . What you comments are exactly right, here the API is just a kind of "push-down" API. We also c

[GitHub] [spark] SparkQA commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
SparkQA commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529989682 **[Test build #110420 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequ

[GitHub] [spark] AmplabJenkins removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529989859 Test PASSed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529989846 Merged build finished. Test PASSed. --

[GitHub] [spark] wangyum commented on issue #25542: [SPARK-28840][SQL] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive

2019-09-10 Thread GitBox
wangyum commented on issue #25542: [SPARK-28840][SQL] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529989874 retest this please

[GitHub] [spark] AmplabJenkins commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529989846 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529989859 Test PASSed. Refer to this link for build results (access rights to CI server needed

[GitHub] [spark] SparkQA removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
SparkQA removed a comment on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-529984790 **[Test build #110420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkP

[GitHub] [spark] wangyum commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file

2019-09-10 Thread GitBox
wangyum commented on issue #25690: [SPARK-27831][FOLLOW-UP][SQL][TEST][test-hadoop3.2][test-maven] Move Hive test jars to local file URL: https://github.com/apache/spark/pull/25690#issuecomment-529990645 It will fail because we _fixed invalid tests in CliSuite_ by [SPARK-28988](https://is

[GitHub] [spark] wangyum commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by H

2019-09-10 Thread GitBox
wangyum commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529991055 retest this please

[GitHub] [spark] AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529991193 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529991234 Merged build finished. Test PASSed. This is an automated messa

[GitHub] [spark] AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529991200 Test PASSed. Refer to this link for build results (access rights to CI server needed): htt

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529991281 Test PASSed. Refer to t

[GitHub] [spark] AmplabJenkins commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529991246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/je

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529991272 Merged build finished. Tes

[GitHub] [spark] AmplabJenkins removed a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529991234 Merged build finished. Test PASSed. This is an automat

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529991281 Test PASSed. Re

[GitHub] [spark] AmplabJenkins removed a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529991246 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkele

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529991272 Merged build finis

[GitHub] [spark] AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529991200 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529991193 Merged build finished. Test PASSed. -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
dongjoon-hyun commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#discussion_r322813610 ## File path: core/src/main/scala/org/apache/spark/metrics/MetricsSystem

[GitHub] [spark] MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type.

2019-09-10 Thread GitBox
MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-529991848 @srowen Thank you for your quick response. > For example, what if I write this type as P

[GitHub] [spark] SparkQA commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
SparkQA commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529992305 **[Test build #110421 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1

[GitHub] [spark] SparkQA commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by H

2019-09-10 Thread GitBox
SparkQA commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529992334 **[Test build #110422 has start

[GitHub] [spark] SparkQA commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
SparkQA commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529992337 **[Test build #110423 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110423/testReport)** for PR 25

[GitHub] [spark] AngersZhuuuu commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AngersZh commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-529993710 > Sorry to chime in late. First of all, I don't think it's a safe or meaningful change at this point. After join hin

[GitHub] [spark] JoshRosen opened a new pull request #25745: [SPARK-29033][SQL][WIP

2019-09-10 Thread GitBox
JoshRosen opened a new pull request #25745: [SPARK-29033][SQL][WIP URL: https://github.com/apache/spark/pull/25745 ### What changes were proposed in this pull request? Spark 2.x has two separate implementations of the "create named struct" expression: regular `CreateNamed

[GitHub] [spark] Gschiavon commented on issue #25728: [SPARK-29020][SQL] Improving array_sort behaviour

2019-09-10 Thread GitBox
Gschiavon commented on issue #25728: [SPARK-29020][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#issuecomment-529994076 > I see, is it that the ordering of nulls is different? > I think we'd at least want the doc of each to comment on the difference

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
dongjoon-hyun commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#discussion_r322817351 ## File path: core/src/main/scala/org/apache/spark/metrics/sink/Prometh

[GitHub] [spark] AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP URL: https://github.com/apache/spark/pull/25745#issuecomment-529995115 Merged build finished. Test PASSed. This is an automated message from the Apache Git Servic

[GitHub] [spark] AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP URL: https://github.com/apache/spark/pull/25745#issuecomment-529995129 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-ma

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995197 Test PASSed. Refer to t

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995185 Merged build finished. Tes

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995197 Test PASSed. Re

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995185 Merged build finis

[GitHub] [spark] koertkuipers commented on issue #25739: [WIP][SPARK-28945][CORE][SQL] Support concurrent dynamic partition writes to different partitions in the same table

2019-09-10 Thread GitBox
t; > For example, when insert overwrite `Partition(day=20190910, hour=01, action)`, Spark will first delete all the partitions under `$some/path/to/table/location/day=20190910/hour=01` when i need to overwrite a particular partition in a filesource such as parquet i will write directl

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995983 Test FAILed. Refer to t

[GitHub] [spark] SparkQA commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
SparkQA commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529996088 **[Test build #110424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/11

[GitHub] [spark] AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is create

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995970 Merged build finished. Tes

[GitHub] [spark] SparkQA commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
SparkQA commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529996090 **[Test build #110425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1

[GitHub] [spark] SparkQA commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by H

2019-09-10 Thread GitBox
SparkQA commented on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995950 **[Test build #110422 has finis

[GitHub] [spark] srowen commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type.

2019-09-10 Thread GitBox
srowen commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-529996045 This is just for my education, but how is TimeType mapped to Parquet TIME here? Yes, that's goo

[GitHub] [spark] AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529995129 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] dongjoon-hyun commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
dongjoon-hyun commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529996193 Yes. Prometheus.io is a CNCF project which is used widely with K8s. - https://prometheus.io --

[GitHub] [spark] AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529995115 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995970 Merged build finis

[GitHub] [spark] SparkQA removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is crea

2019-09-10 Thread GitBox
SparkQA removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529992334 **[Test build #110422 ha

[GitHub] [spark] AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which i

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25542: [SPARK-28840][SQL][test-hadoop3.2] conf.getClassLoader in SparkSQLCLIDriver should be avoided as it returns the UDFClassLoader which is created by Hive URL: https://github.com/apache/spark/pull/25542#issuecomment-529995983 Test FAILed. Re

[GitHub] [spark] srowen commented on issue #25728: [SPARK-29020][SQL] Improving array_sort behaviour

2019-09-10 Thread GitBox
srowen commented on issue #25728: [SPARK-29020][SQL] Improving array_sort behaviour URL: https://github.com/apache/spark/pull/25728#issuecomment-529997046 Yeah, I guess it's unfortunate that the existing null ordering semantics aren't the same, or else these could be unified. Later, maybe

[GitHub] [spark] dongjoon-hyun edited a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
dongjoon-hyun edited a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529996193 Yes. Prometheus.io is a CNCF project which is used widely with K8s. - https://github.com

[GitHub] [spark] srowen commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
srowen commented on a change in pull request #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#discussion_r322820078 ## File path: core/src/main/scala/org/apache/spark/metrics/MetricsSystem.scala

[GitHub] [spark] dongjoon-hyun edited a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
dongjoon-hyun edited a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529996193 Thank you for review, @srowen . Prometheus.io is a CNCF project which is used widely with

[GitHub] [spark] AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529998765 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529998776 Test PASSed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529998855 Test PASSed. Refer to this link for build results (access rights to CI server needed): htt

[GitHub] [spark] AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529998765 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529998844 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529998855 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-529998776 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25741: [WIP][SPARK-29032][CORE] Simplify Prometheus support by adding PrometheusServlet URL: https://github.com/apache/spark/pull/25741#issuecomment-529998844 Merged build finished. Test PASSed. -

[GitHub] [spark] JoshRosen commented on a change in pull request #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
JoshRosen commented on a change in pull request #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#discussion_r322824766 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expr

[GitHub] [spark] planga82 commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
planga82 commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530002277 Thanks for the revision @juliuszsompolski !! This is an automated mess

[GitHub] [spark] srowen commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients

2019-09-10 Thread GitBox
srowen commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-530003220 I'm not sure this is a problem. These are differ

[GitHub] [spark] AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530002291 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] maryannxue edited a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
maryannxue edited a comment on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-530001920 > May be it have some threshold to use Hint, but inject unsafe rule or strategies also cause problems. Not sure if I

[GitHub] [spark] AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530002298 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.e

[GitHub] [spark] SparkQA commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
SparkQA commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530003081 **[Test build #110426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110426/testReport)** for PR 25718

[GitHub] [spark] AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530002291 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530002298 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] maryannxue commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
maryannxue commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-530001920 > May be it have some threshold to use Hint, but inject unsafe rule or strategies also cause problems. Not sure if I understand

[GitHub] [spark] AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530007513 Merged build finished. Test PASSed. This is an automated message

[GitHub] [spark] JoshRosen commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct

2019-09-10 Thread GitBox
JoshRosen commented on issue #25745: [SPARK-29033][SQL][WIP] Always use UnsafeRow-based version of CreateNamedStruct URL: https://github.com/apache/spark/pull/25745#issuecomment-530006567 As a high-level illustration of why I think this _might_ improve performance, compare the following be

[GitHub] [spark] SparkQA commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
SparkQA commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530007400 **[Test build #110426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110426/testReport)** for PR 2571

[GitHub] [spark] AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins commented on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530007522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkin

[GitHub] [spark] SparkQA removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
SparkQA removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530003081 **[Test build #110426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/110426/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530007522 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
AmplabJenkins removed a comment on issue #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#issuecomment-530007513 Merged build finished. Test PASSed. This is an automated

[GitHub] [spark] advancedxy commented on issue #25739: [WIP][SPARK-28945][CORE][SQL] Support concurrent dynamic partition writes to different partitions in the same table

2019-09-10 Thread GitBox
advancedxy commented on issue #25739: [WIP][SPARK-28945][CORE][SQL] Support concurrent dynamic partition writes to different partitions in the same table URL: https://github.com/apache/spark/pull/25739#issuecomment-530009383 > i was not aware that there is an alternative syntax or way of do

[GitHub] [spark] AngersZhuuuu commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
AngersZh commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-530009271 > > May be it have some threshold to use Hint, but inject unsafe rule or strategies also cause problems. > > Not sure if

[GitHub] [spark] wangyum commented on issue #25694: [SPARK-28982][SQL][test-hadoop3.2] Implementation Spark's own GetTypeInfoOperation

2019-09-10 Thread GitBox
wangyum commented on issue #25694: [SPARK-28982][SQL][test-hadoop3.2] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-530012732 Result: ``` Transaction isolation: TRANSACTION_REPEATABLE_READ Beeline version 2.3.6 by Apach

[GitHub] [spark] srowen closed pull request #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
srowen closed pull request #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737 This is an automated message from the Apache Git

[GitHub] [spark] srowen commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
srowen commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-530013298 Merged to master This is an

[GitHub] [spark] MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type.

2019-09-10 Thread GitBox
MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-530013450 > how is TimeType mapped to Parquet TIME here? I think in the same way as `TIMESTAMP`. W

[GitHub] [spark] maryannxue commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint

2019-09-10 Thread GitBox
maryannxue commented on issue #25071: [SPARK-28292][SQL] Enable inject user-defined Hint URL: https://github.com/apache/spark/pull/25071#issuecomment-530014349 > I think safety is not the point we need to concern, but demand. If that were true, there would be no point of having inter

[GitHub] [spark] wangyum closed pull request #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation

2019-09-10 Thread GitBox
wangyum closed pull request #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [spark] wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation

2019-09-10 Thread GitBox
wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-530015322 Merged to master This is an automated message from t

[GitHub] [spark] wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation

2019-09-10 Thread GitBox
wangyum commented on issue #25694: [SPARK-28982][SQL] Implementation Spark's own GetTypeInfoOperation URL: https://github.com/apache/spark/pull/25694#issuecomment-530015245 Thank you @AngersZh and @juliuszsompolski This

[GitHub] [spark] koertkuipers commented on issue #25739: [WIP][SPARK-28945][CORE][SQL] Support concurrent dynamic partition writes to different partitions in the same table

2019-09-10 Thread GitBox
koertkuipers commented on issue #25739: [WIP][SPARK-28945][CORE][SQL] Support concurrent dynamic partition writes to different partitions in the same table URL: https://github.com/apache/spark/pull/25739#issuecomment-530015483 > > i was not aware that there is an alternative syntax or way o

[GitHub] [spark] srowen commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type.

2019-09-10 Thread GitBox
srowen commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-530016931 Just to be clear, does ParquetSchemaConverter need a new case to handle the new type then? to a

[GitHub] [spark] dilipbiswal commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md

2019-09-10 Thread GitBox
dilipbiswal commented on issue #25737: [SPARK-29028][DOCS] Add links to IBM Cloud Object Storage connector in cloud-integration.md URL: https://github.com/apache/spark/pull/25737#issuecomment-530022494 Thank you very much @srowen

[GitHub] [spark] sujith71955 commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients

2019-09-10 Thread GitBox
sujith71955 commented on issue #25720: [SPARK-28930][SQL] Last Access Time value shall display 'UNKNOWN' and 'null' values will be shown in its capitals 'NULL' for all clients URL: https://github.com/apache/spark/pull/25720#issuecomment-530024137 > I'm not sure this is a problem. These are

[GitHub] [spark] MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type.

2019-09-10 Thread GitBox
MaxGekk commented on issue #25678: [SPARK-28973][SQL] Add `TimeType` and support `java.time.LocalTime` as its external type. URL: https://github.com/apache/spark/pull/25678#issuecomment-530024672 > does ParquetSchemaConverter need a new case to handle the new type then? to and from Parquet

[GitHub] [spark] juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#discussion_r322854022 ## File path: docs/web-ui.md ## @@ -369,3 +369,44 @@ The web UI includes a Streaming tab if

[GitHub] [spark] juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#discussion_r322853703 ## File path: docs/web-ui.md ## @@ -369,3 +369,44 @@ The web UI includes a Streaming tab if

[GitHub] [spark] juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab

2019-09-10 Thread GitBox
juliuszsompolski commented on a change in pull request #25718: [SPARK-28373][DOCS][WebUI] JDBC/ODBC Server Tab URL: https://github.com/apache/spark/pull/25718#discussion_r322852888 ## File path: docs/web-ui.md ## @@ -369,3 +369,44 @@ The web UI includes a Streaming tab if

[GitHub] [spark] juliuszsompolski commented on a change in pull request #25723: [SPARK-29019][WebUI] Improve tooltip JDBC/ODBC Server tab

2019-09-10 Thread GitBox
juliuszsompolski commented on a change in pull request #25723: [SPARK-29019][WebUI] Improve tooltip JDBC/ODBC Server tab URL: https://github.com/apache/spark/pull/25723#discussion_r322855913 ## File path: core/src/main/scala/org/apache/spark/ui/UIUtils.scala ## @@ -311,7 +

<    1   2   3   4   5   6   7   8   9   10   >