[GitHub] [spark] SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198847 **[Test build #105585 has started](https://amplab.cs.berkeley.edu

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198573 Merged build finished. Test PASSed. ---

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198582 @dongjoon-hyun Do you think we need test maven on hadoop-2.7? ---

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198573 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198578 Test PASSed. Refer to this link for build results (access

[GitHub] [spark] dbtsai commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dbtsai commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494198592 @cloud-fan This feature was already added in 2.4.0 (by us, Apple). The reason is avro and

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198578 Test PASSed. Refer to this link for build results

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198348 maven on hadoop-3.2 failed: ``` VersionsSuite: 15:41:45.

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494198372 retest this please --

[GitHub] [spark] cloud-fan commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2

2019-05-20 Thread GitBox
cloud-fan commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#discussion_r285815402 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -267,

[GitHub] [spark] cloud-fan commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2

2019-05-20 Thread GitBox
cloud-fan commented on a change in pull request #24233: [SPARK-26356][SQL] remove SaveMode from data source v2 URL: https://github.com/apache/spark/pull/24233#discussion_r285814613 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -56,1

[GitHub] [spark] rdblue commented on a change in pull request #24594: [SPARK-27693][SQL] Add default catalog property

2019-05-20 Thread GitBox
rdblue commented on a change in pull request #24594: [SPARK-27693][SQL] Add default catalog property URL: https://github.com/apache/spark/pull/24594#discussion_r285814039 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1767,6 +

[GitHub] [spark] cloud-fan commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
cloud-fan commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494196717 @dbtsai yea we can specify custom schema during reading, I'm just wondering if it's an

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494196180 Test FAILed. Refer to this link for build results (access rig

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494196176 Merged build finished. Test FAILed. ---

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494196176 Merged build finished. Test FAILed.

[GitHub] [spark] SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494132952 **[Test build #105578 has started](https://amplab.cs.berkeley.edu/jenk

[GitHub] [spark] cloud-fan edited a comment on issue #24565: [SPARK-27665][Core] Split fetch shuffle blocks protocol from OpenBlocks

2019-05-20 Thread GitBox
cloud-fan edited a comment on issue #24565: [SPARK-27665][Core] Split fetch shuffle blocks protocol from OpenBlocks URL: https://github.com/apache/spark/pull/24565#issuecomment-494195958 @squito thanks for opening the ticket! Agree with adding versioning separately. I have no more comments

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494196180 Test FAILed. Refer to this link for build results (access rights to CI

[GitHub] [spark] SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494196019 **[Test build #105578 has finished](https://amplab.cs.berkeley.edu/jenkins/job

[GitHub] [spark] cloud-fan commented on issue #24565: [SPARK-27665][Core] Split fetch shuffle blocks protocol from OpenBlocks

2019-05-20 Thread GitBox
cloud-fan commented on issue #24565: [SPARK-27665][Core] Split fetch shuffle blocks protocol from OpenBlocks URL: https://github.com/apache/spark/pull/24565#issuecomment-494195958 @squito thanks for opening the ticket! Agree with adding versioning separately. I have no more comments on thi

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494195112 Test PASSed. Refer to this link for build results (access rig

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494195110 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494195112 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494195110 Merged build finished. Test PASSed.

[GitHub] [spark] dbtsai commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dbtsai commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494195115 @viirya I think user provided avro schema has to match catalyst schema to avoid any discr

[GitHub] [spark] cloud-fan commented on a change in pull request #24594: [SPARK-27693][SQL] Add default catalog property

2019-05-20 Thread GitBox
cloud-fan commented on a change in pull request #24594: [SPARK-27693][SQL] Add default catalog property URL: https://github.com/apache/spark/pull/24594#discussion_r285812293 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1767,

[GitHub] [spark] SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494164479 **[Test build #105579 has started](https://amplab.cs.berkeley.edu/jenk

[GitHub] [spark] SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494194800 **[Test build #105579 has finished](https://amplab.cs.berkeley.edu/jenkins/job

[GitHub] [spark] AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494192659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berk

[GitHub] [spark] AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494192656 Merged build finished. Test FAILed. This is an aut

[GitHub] [spark] SparkQA removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
SparkQA removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179789 **[Test build #105582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105582/testReport)*

[GitHub] [spark] AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494192489 Test PASSed. Refer to this link for build results (access rights to CI s

[GitHub] [spark] AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494192656 Merged build finished. Test FAILed. This is an automated m

[GitHub] [spark] SparkQA commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
SparkQA commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494192566 **[Test build #105582 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105582/testReport)** for

[GitHub] [spark] AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494192485 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494192659 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu

[GitHub] [spark] AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494192489 Test PASSed. Refer to this link for build results (access rights to CI server ne

[GitHub] [spark] AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494192485 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
SparkQA commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494192258 **[Test build #105584 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPull

[GitHub] [spark] SparkQA removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
SparkQA removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185824 **[Test build #105584 has started](https://amplab.cs.berkeley.edu/jenkins/job/Sp

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dongjoon-hyun commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#discussion_r285805169 ## File path: external/avro/src/test/scala/org/apache/sp

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dongjoon-hyun commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#discussion_r285804966 ## File path: external/avro/src/main/scala/org/apache/sp

[GitHub] [spark] SparkQA commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
SparkQA commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185824 **[Test build #105584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullR

[GitHub] [spark] AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185410 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185413 Test PASSed. Refer to this link for build results (access rights to CI s

[GitHub] [spark] dbtsai commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dbtsai commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#discussion_r285803185 ## File path: external/avro/src/main/scala/org/apache/spark/sql

[GitHub] [spark] AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185413 Test PASSed. Refer to this link for build results (access rights to CI server ne

[GitHub] [spark] AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#issuecomment-494185410 Merged build finished. Test PASSed. --

[GitHub] [spark] dbtsai commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering

2019-05-20 Thread GitBox
dbtsai commented on a change in pull request #24635: [SPARK-27762][SQL] Support user provided avro schema for writing fields with different ordering URL: https://github.com/apache/spark/pull/24635#discussion_r285802930 ## File path: external/avro/src/test/scala/org/apache/spark/sql

[GitHub] [spark] AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494182406 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494182406 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494182409 Test PASSed. Refer to this link for build results (acces

[GitHub] [spark] AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494182409 Test PASSed. Refer to this link for build result

[GitHub] [spark] SparkQA commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
SparkQA commented on issue #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#issuecomment-494181330 **[Test build #105583 has started](https://amplab.cs.berkeley.edu

[GitHub] [spark] skambha commented on a change in pull request #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
skambha commented on a change in pull request #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#discussion_r285798913 ## File path: sql/catalyst/src/main/sc

[GitHub] [spark] skambha commented on a change in pull request #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals

2019-05-20 Thread GitBox
skambha commented on a change in pull request #24593: [SPARK-27692][SQL] Add new optimizer rule to evaluate the deterministic scala udf only once if all inputs are literals URL: https://github.com/apache/spark/pull/24593#discussion_r285798821 ## File path: sql/core/src/test/scala/o

[GitHub] [spark] AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.ber

[GitHub] [spark] SparkQA commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
SparkQA commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179789 **[Test build #105582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105582/testReport)** for P

[GitHub] [spark] AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179394 Merged build finished. Test PASSed. This is an aut

[GitHub] [spark] AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179404 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653#issuecomment-494179394 Merged build finished. Test PASSed. This is an automated m

[GitHub] [spark] maryannxue opened a new pull request #24653: [SPARK-27783][SQL] Add customizable hint error handler

2019-05-20 Thread GitBox
maryannxue opened a new pull request #24653: [SPARK-27783][SQL] Add customizable hint error handler URL: https://github.com/apache/spark/pull/24653 ## What changes were proposed in this pull request? Added an interface for handling hint errors, with a default implementation class th

[GitHub] [spark] AmplabJenkins removed a comment on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#issuecomment-494177802 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#issuecomment-494177808 Test PASSed. Refer to this link for build results (acces

[GitHub] [spark] AmplabJenkins commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#issuecomment-494177808 Test PASSed. Refer to this link for build results (access rights

[GitHub] [spark] AmplabJenkins commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#issuecomment-494177802 Merged build finished. Test PASSed. ---

[GitHub] [spark] SparkQA commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
SparkQA commented on issue #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#issuecomment-494176642 **[Test build #105581 has started](https://amplab.cs.berkeley.edu/jenkins

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494176292 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494176296 Test PASSed. Refer to this link for build results (access rights to CI server need

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494176292 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494176296 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
SparkQA commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494175084 **[Test build #105580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuil

[GitHub] [spark] AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494006901 Can one of the admins verify this patch? ---

[GitHub] [spark] davidnavas commented on a change in pull request #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads

2019-05-20 Thread GitBox
davidnavas commented on a change in pull request #24616: [SPARK-27726] [Core] Fix performance of ElementTrackingStore deletes when using InMemoryStore under high loads URL: https://github.com/apache/spark/pull/24616#discussion_r285792194 ## File path: common/kvstore/src/main/java/

[GitHub] [spark] BryanCutler commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
BryanCutler commented on issue #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#issuecomment-494174622 ok to test This is an automat

[GitHub] [spark] BryanCutler commented on a change in pull request #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions

2019-05-20 Thread GitBox
BryanCutler commented on a change in pull request #24650: [SPARK-27778][PySpark] Fix toPandas conversion using arrow for DFs with no partitions URL: https://github.com/apache/spark/pull/24650#discussion_r285792024 ## File path: python/pyspark/sql/tests/test_arrow.py ## @@

[GitHub] [spark] dilipbiswal commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
dilipbiswal commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494172482 cc @gatorsmile @maryannxue ---

[GitHub] [spark] dongjoon-hyun commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

2019-05-20 Thread GitBox
dongjoon-hyun commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494172156 This is reverted via https://github.com/apache/spark/commit/039db879f476241efd8a6ff17c1f5998fa8187be an

[GitHub] [spark] dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
dongjoon-hyun commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494171638 Thank you for additional one, @wangyum . To sum up, - `sbt` on `ha

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

2019-05-20 Thread GitBox
dongjoon-hyun edited a comment on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494167695 Thank you, @gatorsmile and @hvanhovell . @gatorsmile also gave the regression example about `ex

[GitHub] [spark] dongjoon-hyun commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

2019-05-20 Thread GitBox
dongjoon-hyun commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494167695 Thank you, @gatorsmile and @hvanhovell . @gatorsmile also gave the regression example about `explain o

[GitHub] [spark] wypoon commented on issue #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-05-20 Thread GitBox
wypoon commented on issue #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#issuecomment-494167053 I mentioned that I ran some experiments. I ran a TPC-DS query on a 5TB dataset. Before my changes, running with spark.e

[GitHub] [spark] hvanhovell edited a comment on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

2019-05-20 Thread GitBox
hvanhovell edited a comment on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494165538 @viirya can we just fix this by doing the following: ```scala def explain(extended: Boolean): U

[GitHub] [spark] hvanhovell commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans

2019-05-20 Thread GitBox
hvanhovell commented on issue #24464: [SPARK-27439][SQL] Explainging Dataset should show correct resolved plans URL: https://github.com/apache/spark/pull/24464#issuecomment-494165538 @viirya can we just fix this by doing the following: ```scala --

[GitHub] [spark] wypoon commented on issue #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics.

2019-05-20 Thread GitBox
wypoon commented on issue #23767: [SPARK-26329][CORE] Faster polling of executor memory metrics. URL: https://github.com/apache/spark/pull/23767#issuecomment-494165380 @squito I have implemented your suggestions. Also, @squito and @edwinalu, I ran some experiments. Previously, I'd run e

[GitHub] [spark] attilapiros commented on a change in pull request #24499: [SPARK-27677][Core] Serve local disk persisted blocks by the external service after releasing executor by dynamic allocation

2019-05-20 Thread GitBox
attilapiros commented on a change in pull request #24499: [SPARK-27677][Core] Serve local disk persisted blocks by the external service after releasing executor by dynamic allocation URL: https://github.com/apache/spark/pull/24499#discussion_r285779169 ## File path: core/src/main/

[GitHub] [spark] SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
SparkQA commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494164479 **[Test build #105579 has started](https://amplab.cs.berkeley.edu/jenkins/job/

[GitHub] [spark] attilapiros commented on a change in pull request #24499: [SPARK-27677][Core] Serve local disk persisted blocks by the external service after releasing executor by dynamic allocation

2019-05-20 Thread GitBox
attilapiros commented on a change in pull request #24499: [SPARK-27677][Core] Serve local disk persisted blocks by the external service after releasing executor by dynamic allocation URL: https://github.com/apache/spark/pull/24499#discussion_r285781962 ## File path: common/network

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494163925 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494163931 Test PASSed. Refer to this link for build results (access rig

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494163931 Test PASSed. Refer to this link for build results (access rights to CI

[GitHub] [spark] AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494163925 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494162779 Test PASSed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
AmplabJenkins removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494162775 Merged build finished. Test PASSed. -

[GitHub] [spark] AmplabJenkins commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494162779 Test PASSed. Refer to this link for build results (access rights to CI serv

[GitHub] [spark] AmplabJenkins commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
AmplabJenkins commented on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494162775 Merged build finished. Test PASSed. -

[GitHub] [spark] SparkQA removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator

2019-05-20 Thread GitBox
SparkQA removed a comment on issue #24652: [SPARK-27782][SQL] Use '#' to mark expression id embedded in the name field of SubqueryExec operator URL: https://github.com/apache/spark/pull/24652#issuecomment-494107836 **[Test build #105577 has started](https://amplab.cs.berkeley.edu/jenkins/j

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494162525 retest this please ---

[GitHub] [spark] wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile

2019-05-20 Thread GitBox
wangyum commented on issue #24620: [SPARK-27737][SQL][test-hadoop3.2][test-maven] Upgrade to Hive 2.3.5 for Hive Metastore Client and Hadoop-3.2 profile URL: https://github.com/apache/spark/pull/24620#issuecomment-494162383 Hive 2.3.5 should support test with `sbt`. --

<    1   2   3   4   5   6   7   8   >