[GitHub] [spark] SparkQA removed a comment on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

SparkQA removed a comment on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733576388 **[Test build #131763 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131763/testReport)** for PR 30497 at commit

[GitHub] [spark] SparkQA commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

SparkQA commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733695990 **[Test build #131775 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131775/testReport)** for PR 30483 at commit

[GitHub] [spark] SparkQA commented on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

SparkQA commented on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-73376 **[Test build #131776 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131776/testReport)** for PR 30497 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

HyukjinKwon commented on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733699654 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530365453 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -720,10 +720,12 @@ case class

[GitHub] [spark] LuciferYang commented on a change in pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

LuciferYang commented on a change in pull request #30483: URL: https://github.com/apache/spark/pull/30483#discussion_r530370659 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/SpecificParquetRecordReaderBase.java ## @@ -102,13 +105,13

[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530370757 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/window_part3.sql.out ## @@ -71,7 +71,8 @@ insert into datetimes values --

[GitHub] [spark] SparkQA commented on pull request #29950: [SPARK-32945][SQL] Avoid collapsing projects if reaching max allowed common exprs

SparkQA commented on pull request #29950: URL: https://github.com/apache/spark/pull/29950#issuecomment-733728541 **[Test build #131780 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131780/testReport)** for PR 29950 at commit

[GitHub] [spark] gengliangwang commented on pull request #30495: [SPARK-33548][WEBUI] display the jvm peak memory usage on the executor ui

gengliangwang commented on pull request #30495: URL: https://github.com/apache/spark/pull/30495#issuecomment-733748301 @JQ-Cao These metrics are already on the executor page. They will show up after select the additional metrics checkbox:

[GitHub] [spark] cloud-fan closed pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

cloud-fan closed pull request #28026: URL: https://github.com/apache/spark/pull/28026 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r530459131 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -299,13 +301,16 @@ class AstBuilder extends

[GitHub] [spark] SparkQA commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

SparkQA commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733782603 **[Test build #131772 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131772/testReport)** for PR 30440 at commit

[GitHub] [spark] steveloughran commented on pull request #30366: [SPARK-33440][CORE] Use current timestamp with warning log when the issue date for token is not set up properly

steveloughran commented on pull request #30366: URL: https://github.com/apache/spark/pull/30366#issuecomment-733784961 I think you should also log when the issueDate is > the time on the local machine. As discussed, fixing up is hard, but at least printing out "clock mismatch" is a useful

[GitHub] [spark] SparkQA commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

SparkQA commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733789888 **[Test build #131770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131770/testReport)** for PR 29729 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r530471201 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -3718,6 +3718,49 @@ class SQLQuerySuite extends QueryTest with

[GitHub] [spark] SparkQA removed a comment on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

SparkQA removed a comment on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733639162 **[Test build #131770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131770/testReport)** for PR 29729 at commit

[GitHub] [spark] maryannxue commented on a change in pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

maryannxue commented on a change in pull request #30494: URL: https://github.com/apache/spark/pull/30494#discussion_r530490828 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -55,6 +55,11 @@ import

[GitHub] [spark] gaborgsomogyi edited a comment on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

gaborgsomogyi edited a comment on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733652656 @zsxwing @viirya @HeartSaVioR @xuanyuanking I've added the asked change and I would like to you ask to have a look please. Since it's quite a heavy change

[GitHub] [spark] cloud-fan commented on a change in pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

cloud-fan commented on a change in pull request #30494: URL: https://github.com/apache/spark/pull/30494#discussion_r530339227 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -55,6 +55,11 @@ import

[GitHub] [spark] AmplabJenkins commented on pull request #30497: [SPARK-33540][SQL] Subexpression elimination for interpreted predicate

AmplabJenkins commented on pull request #30497: URL: https://github.com/apache/spark/pull/30497#issuecomment-733694231 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

AmplabJenkins commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733694233 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

SparkQA commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733713063 **[Test build #131769 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131769/testReport)** for PR 30486 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

AmplabJenkins commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733735868 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

SparkQA commented on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733736073 **[Test build #131781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131781/testReport)** for PR 30499 at commit

[GitHub] [spark] SparkQA commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

SparkQA commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733785521 **[Test build #131771 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131771/testReport)** for PR 30483 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

AmplabJenkins commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733810304 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

SparkQA commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733819063 **[Test build #131777 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131777/testReport)** for PR 30440 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

SparkQA removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733700290 **[Test build #131777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131777/testReport)** for PR 30440 at commit

[GitHub] [spark] SparkQA commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

SparkQA commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733700290 **[Test build #131777 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131777/testReport)** for PR 30440 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530366120 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1254,10 +1260,11 @@ abstract class CastBase

[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

LuciferYang commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r530396435 ## File path: pom.xml ## @@ -164,6 +164,7 @@ 3.2.2 2.12.10 2.12 +-Ywarn-unused-import Review comment: @HyukjinKwon

[GitHub] [spark] WinkerDu commented on pull request #29000: [SPARK-27194][SPARK-29302][SQL] Fix commit collision in dynamic partition overwrite mode

WinkerDu commented on pull request #29000: URL: https://github.com/apache/spark/pull/29000#issuecomment-733725405 @cloud-fan yes, I have a JIRA account named 'duripeng' This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

HyukjinKwon commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r530422613 ## File path: pom.xml ## @@ -164,6 +164,7 @@ 3.2.2 2.12.10 2.12 +-Ywarn-unused-import Review comment: Thanks @LuciferYang

[GitHub] [spark] SparkQA commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

SparkQA commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733763576 **[Test build #131782 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131782/testReport)** for PR 30483 at commit

[GitHub] [spark] SparkQA commented on pull request #30398: [WIP][SPARK-33452][SQL] Create V2 SHOW PARTITIONS execution node

SparkQA commented on pull request #30398: URL: https://github.com/apache/spark/pull/30398#issuecomment-733763807 **[Test build #131783 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131783/testReport)** for PR 30398 at commit

[GitHub] [spark] cloud-fan commented on pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

cloud-fan commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733763818 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

AmplabJenkins removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733802592 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

AmplabJenkins commented on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733802592 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

AmplabJenkins removed a comment on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733810304 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

AmplabJenkins removed a comment on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-733510491 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] zero323 commented on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

zero323 commented on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733666568 > cc @zero323 in case you're interested in it. We might have to add the R version as well with a separate JIRA. Thanks @HyukjinKwon . I'll take care of that, once this PR

[GitHub] [spark] AmplabJenkins commented on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables

AmplabJenkins commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-733666293 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29122: [SPARK-32320][PYSPARK] Remove mutable default arguments

AmplabJenkins commented on pull request #29122: URL: https://github.com/apache/spark/pull/29122#issuecomment-733666279 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30479: [SPARK-33527][SQL] Extend the function of decode so as consistent with mainstream databases

AmplabJenkins commented on pull request #30479: URL: https://github.com/apache/spark/pull/30479#issuecomment-733666278 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30408: [SPARK-33477][SQL] Hive Metastore support filter by date type

AmplabJenkins commented on pull request #30408: URL: https://github.com/apache/spark/pull/30408#issuecomment-733666327 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30289: [SPARK-33141][SQL] Capture SQL configs when creating permanent views

AmplabJenkins commented on pull request #30289: URL: https://github.com/apache/spark/pull/30289#issuecomment-733666280 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

AmplabJenkins removed a comment on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733670353 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

HyukjinKwon commented on a change in pull request #30498: URL: https://github.com/apache/spark/pull/30498#discussion_r530284077 ## File path: mllib/src/main/scala/org/apache/spark/ml/functions.scala ## @@ -72,6 +72,21 @@ object functions { } } + private val

[GitHub] [spark] SparkQA removed a comment on pull request #30499: [WIP][SPARK-33558][SQL][TESTS] Unify v1 and v2 ALTER TABLE .. PARTITION tests

SparkQA removed a comment on pull request #30499: URL: https://github.com/apache/spark/pull/30499#issuecomment-733667386 **[Test build #131773 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131773/testReport)** for PR 30499 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30289: [SPARK-33141][SQL] Capture SQL configs when creating permanent views

AmplabJenkins commented on pull request #30289: URL: https://github.com/apache/spark/pull/30289#issuecomment-733671141 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30498: [SPARK-33556][ML] Add array_to_vector function for dataframe column

AmplabJenkins commented on pull request #30498: URL: https://github.com/apache/spark/pull/30498#issuecomment-733674462 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

AmplabJenkins commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733694247 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

AmplabJenkins commented on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733694235 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

AmplabJenkins commented on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-733694230 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #30289: [SPARK-33141][SQL] Capture SQL configs when creating permanent views

AmplabJenkins commented on pull request #30289: URL: https://github.com/apache/spark/pull/30289#issuecomment-733694234 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530364835 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -364,6 +364,15 @@ object DateTimeUtils { }

[GitHub] [spark] cloud-fan commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

cloud-fan commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r530367089 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -945,6 +934,36 @@ abstract class

[GitHub] [spark] SparkQA removed a comment on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

SparkQA removed a comment on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733638740 **[Test build #131768 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131768/testReport)** for PR 30492 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

Ngone51 commented on a change in pull request #30488: URL: https://github.com/apache/spark/pull/30488#discussion_r530377805 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -231,7 +229,8 @@ class Dataset[T] private[sql]( case _ =>

[GitHub] [spark] SparkQA commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

SparkQA commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733709732 **[Test build #131768 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131768/testReport)** for PR 30492 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30486: [SPARK-33530][CORE] Support --archives and spark.archives option natively

SparkQA removed a comment on pull request #30486: URL: https://github.com/apache/spark/pull/30486#issuecomment-733639558 **[Test build #131769 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131769/testReport)** for PR 30486 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

SparkQA removed a comment on pull request #30440: URL: https://github.com/apache/spark/pull/30440#issuecomment-733576716 **[Test build #131764 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131764/testReport)** for PR 30440 at commit

[GitHub] [spark] SparkQA commented on pull request #30483: [WIP][SPARK-33449][SQL] Add File Metadata cache support for Parquet and Orc

SparkQA commented on pull request #30483: URL: https://github.com/apache/spark/pull/30483#issuecomment-733720524 **[Test build #131760 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131760/testReport)** for PR 30483 at commit

[GitHub] [spark] SparkQA commented on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

SparkQA commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733724342 **[Test build #131762 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131762/testReport)** for PR 28026 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28026: [SPARK-31257][SQL] Unify create table syntax

SparkQA removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733569564 **[Test build #131762 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131762/testReport)** for PR 28026 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

cloud-fan commented on a change in pull request #30488: URL: https://github.com/apache/spark/pull/30488#discussion_r530424929 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/namedExpressions.scala ## @@ -178,7 +179,17 @@ case class

[GitHub] [spark] SparkQA commented on pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

SparkQA commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733764803 **[Test build #131784 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131784/testReport)** for PR 28026 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r530475127 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -3104,6 +3106,66 @@ class Analyzer(override val

[GitHub] [spark] AmplabJenkins commented on pull request #29729: [SPARK-32032][SS] Avoid infinite wait in driver because of KafkaConsumer.poll(long) API

AmplabJenkins commented on pull request #29729: URL: https://github.com/apache/spark/pull/29729#issuecomment-733798910 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

AmplabJenkins commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733798906 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

cloud-fan commented on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-733680528 It's unfortunate that we can't retain the shuffle origin info when we optimize out the repartition shuffle. The approach here looks like a good compromise.

[GitHub] [spark] cloud-fan commented on a change in pull request #30440: [SPARK-33496][SQL]Improve error message of ANSI explicit cast

cloud-fan commented on a change in pull request #30440: URL: https://github.com/apache/spark/pull/30440#discussion_r530360448 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1876,6 +1894,35 @@ object AnsiCast { case

[GitHub] [spark] Ngone51 commented on pull request #30488: [SPARK-33071][SPARK-33536][SQL] Avoid changing dataset_id of LogicalPlan in join() to not break DetectAmbiguousSelfJoin

Ngone51 commented on pull request #30488: URL: https://github.com/apache/spark/pull/30488#issuecomment-733709372 Yeah...I fixed it just now. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun commented on pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

dongjoon-hyun commented on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733831187 Hi, @cloud-fan and @rdblue . It's good to have this, but this seems to break Scala 2.13 compilation. cc @srowen and @HyukjinKwon

[GitHub] [spark] mridulm edited a comment on pull request #30446: [spark-33504] The application log in the Spark history server contains sensitive attributes should be redated

mridulm edited a comment on pull request #30446: URL: https://github.com/apache/spark/pull/30446#issuecomment-733832829 @tgravescs The question I had was regarding when redaction is applied - while logging the event or when surfacing in UI/cli/etc. Only for the case of

[GitHub] [spark] mridulm edited a comment on pull request #30446: [spark-33504] The application log in the Spark history server contains sensitive attributes should be redated

mridulm edited a comment on pull request #30446: URL: https://github.com/apache/spark/pull/30446#issuecomment-733832829 @tgravescs The question I had was regarding when redaction is applied - while logging the event or when surfacing in UI/cli/etc. Only for the case of

[GitHub] [spark] mridulm edited a comment on pull request #30446: [spark-33504] The application log in the Spark history server contains sensitive attributes should be redated

mridulm edited a comment on pull request #30446: URL: https://github.com/apache/spark/pull/30446#issuecomment-733832829 @tgravescs The question I had was regarding when redaction is applied - while logging the event or when surfacing in UI/cli/etc. Only for the case of

[GitHub] [spark] yaooqinn commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

yaooqinn commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733848715 I will address the comment for the main code change first but leave the test part tomorrow. It’s too late now :)

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28026: [SPARK-31257][SPARK-33561][SQL] Unify create table syntax

AmplabJenkins removed a comment on pull request #28026: URL: https://github.com/apache/spark/pull/28026#issuecomment-733866206 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

dongjoon-hyun commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733866284 Could you review this PR once more please, @viirya ? This is an automated message from the Apache Git

[GitHub] [spark] SparkQA commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

SparkQA commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733868465 **[Test build #131796 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131796/testReport)** for PR 29893 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #30504: [SPARK-33544][SQL] Optimizer should not insert filter when when explode with CreateArray/CreateMap

dongjoon-hyun commented on pull request #30504: URL: https://github.com/apache/spark/pull/30504#issuecomment-733903138 Also, cc @maropu , too. This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] holdenk commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

holdenk commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733903026 Hi @dongjoon-hyun, I'm taking this week away from open source to take my puppy to go see snow for the first time. I'll do a review on Monday. Thanks for understanding :)

[GitHub] [spark] SparkQA commented on pull request #30398: [SPARK-33452][SQL] Support v2 SHOW PARTITIONS

SparkQA commented on pull request #30398: URL: https://github.com/apache/spark/pull/30398#issuecomment-733905430 **[Test build #131783 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131783/testReport)** for PR 30398 at commit

[GitHub] [spark] SparkQA commented on pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables

SparkQA commented on pull request #30403: URL: https://github.com/apache/spark/pull/30403#issuecomment-733905643 **[Test build #131799 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131799/testReport)** for PR 30403 at commit

[GitHub] [spark] tgravescs commented on a change in pull request #30504: [SPARK-33544][SQL] Optimizer should not insert filter when explode with CreateArray/CreateMap

tgravescs commented on a change in pull request #30504: URL: https://github.com/apache/spark/pull/30504#discussion_r530614422 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -873,24 +873,30 @@ object

[GitHub] [spark] tgravescs commented on a change in pull request #30504: [SPARK-33544][SQL] Optimizer should not insert filter when explode with CreateArray/CreateMap

tgravescs commented on a change in pull request #30504: URL: https://github.com/apache/spark/pull/30504#discussion_r530614422 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -873,24 +873,30 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30502: [SPARK-31257][SPARK-33561][SQL][FOLLOWUP] Fix Scala 2.13 compilation

AmplabJenkins removed a comment on pull request #30502: URL: https://github.com/apache/spark/pull/30502#issuecomment-733946528 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30502: [SPARK-31257][SPARK-33561][SQL][FOLLOWUP] Fix Scala 2.13 compilation

AmplabJenkins commented on pull request #30502: URL: https://github.com/apache/spark/pull/30502#issuecomment-733946528 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Tagar commented on pull request #30492: [SPARK-33545][CORE] Support Fallback Storage during Worker decommission

Tagar commented on pull request #30492: URL: https://github.com/apache/spark/pull/30492#issuecomment-733830182 @dongjoon-hyun is this only for shuffled data? I was wondering if it would also be possible to cover `MEMORY_AND_DISK` for cached dataframes? Thanks!

[GitHub] [spark] dongjoon-hyun opened a new pull request #30502: [SPARK-31257][SPARK-33561][SQL][FOLLOWUP] Fix Scala 2.13 compilation

dongjoon-hyun opened a new pull request #30502: URL: https://github.com/apache/spark/pull/30502 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] xuanyuanking commented on a change in pull request #30395: [SPARK-32863][SS] Full outer stream-stream join

xuanyuanking commented on a change in pull request #30395: URL: https://github.com/apache/spark/pull/30395#discussion_r530517789 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala ## @@ -315,40 +321,17 @@ object

[GitHub] [spark] SparkQA commented on pull request #29066: [SPARK-23889][SQL] DataSourceV2: required sorting and clustering for writes

SparkQA commented on pull request #29066: URL: https://github.com/apache/spark/pull/29066#issuecomment-733843593 **[Test build #131791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131791/testReport)** for PR 29066 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

AmplabJenkins commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-733899415 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30398: [SPARK-33452][SQL] Support v2 SHOW PARTITIONS

SparkQA commented on pull request #30398: URL: https://github.com/apache/spark/pull/30398#issuecomment-733899393 **[Test build #131805 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131805/testReport)** for PR 30398 at commit

[GitHub] [spark] Victsm commented on a change in pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

Victsm commented on a change in pull request #30312: URL: https://github.com/apache/spark/pull/30312#discussion_r530575752 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1992,4 +1992,32 @@ package object config {

[GitHub] [spark] SparkQA commented on pull request #30504: [SPARK-33544][SQL] Optimizer should not insert filter when when explode with CreateArray/CreateMap

SparkQA commented on pull request #30504: URL: https://github.com/apache/spark/pull/30504#issuecomment-733902999 **[Test build #131806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131806/testReport)** for PR 30504 at commit

[GitHub] [spark] otterc commented on a change in pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

otterc commented on a change in pull request #30312: URL: https://github.com/apache/spark/pull/30312#discussion_r530614146 ## File path: core/src/main/scala/org/apache/spark/shuffle/ShuffleBlockPusher.scala ## @@ -0,0 +1,462 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

SparkQA commented on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-733942863 **[Test build #131795 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131795/testReport)** for PR 30312 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30312: [SPARK-32917][SHUFFLE][CORE] Adds support for executors to push shuffle blocks after successful map task completion

SparkQA removed a comment on pull request #30312: URL: https://github.com/apache/spark/pull/30312#issuecomment-733868083 **[Test build #131795 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131795/testReport)** for PR 30312 at commit

<    1   2   3   4   5   6   7   8   9   >