[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

2020-11-25 Thread GitBox
cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530367654 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -945,6 +934,36 @@ abstract class

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

2020-11-25 Thread GitBox
HyukjinKwon commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r530383072 ## File path: pom.xml ## @@ -164,6 +164,7 @@ 3.2.2 2.12.10 2.12 +-Ywarn-unused-import Review comment: @LuciferYang, just

[GitHub] [spark] cloud-fan commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

2020-11-25 Thread GitBox
cloud-fan commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-733723789 @maropu @dongjoon-hyun any more comments? This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733761484 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733761478 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30472: [SPARK-32221][k8s] Avoid possible errors due to incorrect file size or type supplied in spark conf.

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30472: URL: https://github.com/apache/spark/pull/30472#issuecomment-733761479 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30478: [SPARK-33525][SQL] Update hive-service-rpc to 3.1.2

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30478: URL: https://github.com/apache/spark/pull/30478#issuecomment-733761485 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733761481 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733761482 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733761484 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r530465028 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statements.scala ## @@ -290,6 +291,7 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #30430: [SPARK-33503][SQL] Refactor SortOrder class to allow multiple childrens

2020-11-25 Thread GitBox
cloud-fan commented on a change in pull request #30430: URL: https://github.com/apache/spark/pull/30430#discussion_r530478275 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/SortMergeJoinExec.scala ## @@ -68,9 +68,9 @@ case class SortMergeJoinExec(

[GitHub] [spark] mridulm commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
mridulm commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733820509 Thanks for working on this @HyukjinKwon ! I have not taken a very detailed look, but wanted to understand the interaction with use of distributed cache in yarn. How does

[GitHub] [spark] SparkQA removed a comment on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733577461 **[Test build #131765 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131765/testReport)** for PR 29122 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables

2020-11-25 Thread GitBox
cloud-fan commented on a change in pull request #30403: URL: https://github.com/apache/spark/pull/30403#discussion_r530337419 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala ## @@ -442,19 +442,11 @@ class

[GitHub] [spark] xuanyuanking commented on pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

2020-11-25 Thread GitBox
xuanyuanking commented on pull request #30488: URL: https://github.com/apache/spark/pull/30488#issuecomment-733692855 Seems the failed UT is related. ``` org.apache.spark.sql.DataFrameSelfJoinSuite.SPARK-28344: don't fail if there is no ambiguous self join ```

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-733694230 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733694233 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733694231 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733694235 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733694232 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30289: [SPARK-33141][SQL] Capture SQL configs when creating permanent views

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30289: URL: https://github.com/apache/spark/pull/30289#issuecomment-733671141 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] maropu commented on pull request #29950: [SPARK-32945][SQL] Avoid collapsing projects if reaching max allowed common exprs

2020-11-25 Thread GitBox
maropu commented on pull request #29950: URL: https://github.com/apache/spark/pull/29950#issuecomment-733718994 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
SparkQA commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733718755 **[Test build #131764 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131764/testReport)** for PR 30440 at commit

[GitHub] [spark] SparkQA commented on pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

2020-11-25 Thread GitBox
SparkQA commented on pull request #30488: URL: https://github.com/apache/spark/pull/30488#issuecomment-733728016 **[Test build #131778 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131778/testReport)** for PR 30488 at commit

[GitHub] [spark] SparkQA commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
SparkQA commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733728143 **[Test build #131779 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131779/testReport)** for PR 30483 at commit

[GitHub] [spark] LuciferYang commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
LuciferYang commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r530426279 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileMeta.scala ## @@ -0,0 +1,48 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
SparkQA commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733749615 **[Test build #131766 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131766/testReport)** for PR 29893 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733606715 **[Test build #131766 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131766/testReport)** for PR 29893 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733761481 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733761478 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733761482 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30472: [SPARK-32221][k8s] Avoid possible errors due to incorrect file size or type supplied in spark conf.

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30472: URL: https://github.com/apache/spark/pull/30472#issuecomment-733761479 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30478: [SPARK-33525][SQL] Update hive-service-rpc to 3.1.2

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30478: URL: https://github.com/apache/spark/pull/30478#issuecomment-733761485 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733774484 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r530471201 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -3718,6 +3718,49 @@ class SQLQuerySuite extends QueryTest with

[GitHub] [spark] AmplabJenkins commented on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733811005 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733670191 **[Test build #131774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131774/testReport)** for PR 29729 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733674462 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-11-25 Thread GitBox
SparkQA commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733810958 **[Test build #131774 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131774/testReport)** for PR 29729 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733811342 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

2020-11-25 Thread GitBox
SparkQA commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-733811335 **[Test build #131786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131786/testReport)** for PR 30412 at commit

[GitHub] [spark] cloud-fan closed pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
cloud-fan closed pull request #29000: URL: https://github.com/apache/spark/pull/29000 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
cloud-fan commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733687298 GA passed, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

2020-11-25 Thread GitBox
SparkQA commented on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733692060 **[Test build #131763 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131763/testReport)** for PR 30497 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733726093 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30496: [SPARK-33547][SQL] Add usage of typed literal in doc

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30496: URL: https://github.com/apache/spark/pull/30496#issuecomment-733726089 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733726088 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733726096 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733726094 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30472: [SPARK-32221][k8s] Avoid possible errors due to incorrect file size or type supplied in spark conf.

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30472: URL: https://github.com/apache/spark/pull/30472#issuecomment-733726090 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733726087 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733726094 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733726091 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733726096 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733726087 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30472: [SPARK-32221][k8s] Avoid possible errors due to incorrect file size or type supplied in spark conf.

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30472: URL: https://github.com/apache/spark/pull/30472#issuecomment-733577468 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30496: [SPARK-33547][SQL] Add usage of typed literal in doc

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30496: URL: https://github.com/apache/spark/pull/30496#issuecomment-733543113 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733726088 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733402520 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733725927 thank all for patch review! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] LuciferYang commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
LuciferYang commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r530370659 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java ## @@ -102,13 +105,13

[GitHub] [spark] cloud-fan commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

2020-11-25 Thread GitBox
cloud-fan commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733763043 This PR is definitely not related to Spark R. Since jenkins is flaky now, I'm merging it first. Will keep watching the jenkins status.

[GitHub] [spark] gengliangwang closed pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
gengliangwang closed pull request #30440: URL: https://github.com/apache/spark/pull/30440 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733643830 **[Test build #131772 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131772/testReport)** for PR 30440 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733641323 **[Test build #131771 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131771/testReport)** for PR 30483 at commit

[GitHub] [spark] SparkQA commented on pull request #30430: [SPARK-33503][SQL] Refactor SortOrder class to allow multiple childrens

2020-11-25 Thread GitBox
SparkQA commented on pull request #30430: URL: https://github.com/apache/spark/pull/30430#issuecomment-733801134 **[Test build #131785 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131785/testReport)** for PR 30430 at commit

[GitHub] [spark] SparkQA commented on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

2020-11-25 Thread GitBox
SparkQA commented on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733663296 **[Test build #131765 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131765/testReport)** for PR 29122 at commit

[GitHub] [spark] SparkQA commented on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

2020-11-25 Thread GitBox
SparkQA commented on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733667386 **[Test build #131773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131773/testReport)** for PR 30499 at commit

[GitHub] [spark] SparkQA commented on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

2020-11-25 Thread GitBox
SparkQA commented on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733673928 **[Test build #131767 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131767/testReport)** for PR 30498 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733638560 **[Test build #131767 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131767/testReport)** for PR 30498 at commit

[GitHub] [spark] LuciferYang commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
LuciferYang commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r530370659 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java ## @@ -102,13 +105,13

[GitHub] [spark] Ngone51 commented on a change in pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

2020-11-25 Thread GitBox
Ngone51 commented on a change in pull request #30488: URL: https://github.com/apache/spark/pull/30488#discussion_r530378429 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -178,7 +179,17 @@ case class

[GitHub] [spark] SparkQA removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
SparkQA removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733568636 **[Test build #131760 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131760/testReport)** for PR 30483 at commit

[GitHub] [spark] cloud-fan commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

2020-11-25 Thread GitBox
cloud-fan commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733746002 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] gengliangwang commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
gengliangwang commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733767295 @maropu @cloud-fan Thanks for the review! Merging to master This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733798903 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733735868 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733798908 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733798904 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733798906 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733798908 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733798903 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733798922 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733798905 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733798910 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30479: [SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-733666278 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733666279 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-733666283 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733666812 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733572124 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30408: [SPARK-33477][SQL] Hive Metastore support filter by date type

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30408: URL: https://github.com/apache/spark/pull/30408#issuecomment-733666327 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30289: [SPARK-33141][SQL] Capture SQL configs when creating permanent views

2020-11-25 Thread GitBox
AmplabJenkins removed a comment on pull request #30289: URL: https://github.com/apache/spark/pull/30289#issuecomment-733666280 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

2020-11-25 Thread GitBox
AmplabJenkins commented on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733670353 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

2020-11-25 Thread GitBox
SparkQA commented on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733670331 **[Test build #131773 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131773/testReport)** for PR 30499 at commit

[GitHub] [spark] SparkQA commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

2020-11-25 Thread GitBox
SparkQA commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733670191 **[Test build #131774 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131774/testReport)** for PR 29729 at commit

[GitHub] [spark] cloud-fan commented on pull request #30452: [SPARK-33509][SQL] List partition by names from a V2 table which supports partition management

2020-11-25 Thread GitBox
cloud-fan commented on pull request #30452: URL: https://github.com/apache/spark/pull/30452#issuecomment-733683029 I'm merging it to unblock the following work. @stczwd @rdblue please leave comments if you have any concerns, so that we can address them.

[GitHub] [spark] cloud-fan closed pull request #30452: [SPARK-33509][SQL] List partition by names from a V2 table which supports partition management

2020-11-25 Thread GitBox
cloud-fan closed pull request #30452: URL: https://github.com/apache/spark/pull/30452 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30452: [SPARK-33509][SQL] List partition by names from a V2 table which supports partition management

2020-11-25 Thread GitBox
cloud-fan commented on pull request #30452: URL: https://github.com/apache/spark/pull/30452#issuecomment-733683326 merging to master, thanks! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

2020-11-25 Thread GitBox
cloud-fan commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733689087 @WinkerDu do you have a JIRA account? This is an automated message from the Apache Git Service. To respond to

  1   2   3   4   5   6   7   8   9   >