[GitHub] [spark] AmplabJenkins removed a comment on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

AmplabJenkins removed a comment on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866086671 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44673/

[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866086617 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44673/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

AmplabJenkins commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866086671 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44673/ --

[GitHub] [spark] SparkQA commented on pull request #32810: [SPARK-35672][CORE][YARN] Pass user classpath entries to executors using config instead of command line.

SparkQA commented on pull request #32810: URL: https://github.com/apache/spark/pull/32810#issuecomment-866086149 **[Test build #140150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140150/testReport)** for PR 32810 at commit

[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866085992 **[Test build #140149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140149/testReport)** for PR 32940 at commit

[GitHub] [spark] SparkQA commented on pull request #32969: [SPARK-35817][SQL] Restore performance of queries against wide Avro tables

SparkQA commented on pull request #32969: URL: https://github.com/apache/spark/pull/32969#issuecomment-866085922 **[Test build #140148 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140148/testReport)** for PR 32969 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

AmplabJenkins removed a comment on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866084912 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44671/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins removed a comment on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866084915 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44670/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

AmplabJenkins removed a comment on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-866084914 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140142/

[GitHub] [spark] AmplabJenkins commented on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

AmplabJenkins commented on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866084912 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44671/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866084915 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44670/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

AmplabJenkins commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-866084914 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140142/ -- This

[GitHub] [spark] SparkQA commented on pull request #32985: [SPARK-35777][SQL] Check all year-month interval types in UDF

SparkQA commented on pull request #32985: URL: https://github.com/apache/spark/pull/32985#issuecomment-866084458 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44672/ -- This is an automated message from the Apache

[GitHub] [spark] Peng-Lei commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

Peng-Lei commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656318122 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] bersprockets commented on a change in pull request #32969: [SPARK-35817][SQL] Restore performance of queries against wide Avro tables

bersprockets commented on a change in pull request #32969: URL: https://github.com/apache/spark/pull/32969#discussion_r656329916 ## File path: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroUtils.scala ## @@ -202,34 +203,40 @@ private[sql] object AvroUtils extends

[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866077512 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44673/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

SparkQA commented on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866077325 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44671/ -- This is an automated message from the

[GitHub] [spark] mridulm commented on a change in pull request #32286: [SPARK-35181][CORE] Use zstd for spark.io.compression.codec by default

mridulm commented on a change in pull request #32286: URL: https://github.com/apache/spark/pull/32286#discussion_r656324813 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/CoalesceShufflePartitionsSuite.scala ## @@ -66,6 +66,7 @@ class

[GitHub] [spark] SparkQA removed a comment on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

SparkQA removed a comment on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-865985292 **[Test build #140142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140142/testReport)** for PR 33022 at commit

[GitHub] [spark] SparkQA commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

SparkQA commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-866071895 **[Test build #140142 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140142/testReport)** for PR 33022 at commit

[GitHub] [spark] SparkQA commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866071620 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44670/ --

[GitHub] [spark] SparkQA commented on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

SparkQA commented on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866069071 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44671/ -- This is an automated message from the Apache

[GitHub] [spark] Peng-Lei commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

Peng-Lei commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656318122 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] wankunde commented on a change in pull request #32868: [SPARK-35714][CORE] Bug fix for deadlock during the executor shutdown

wankunde commented on a change in pull request #32868: URL: https://github.com/apache/spark/pull/32868#discussion_r656314070 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/WorkerWatcher.scala ## @@ -45,7 +50,14 @@ private[spark] class WorkerWatcher(

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656309080 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656306617 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656306442 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656305853 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -1961,12 +1961,118 @@ class DataSourceV2SQLSuite

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656303414 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowCreateTableExec.scala ## @@ -0,0 +1,120 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656302859 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowCreateTableExec.scala ## @@ -0,0 +1,120 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656301244 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala ## @@ -938,6 +938,10 @@ private[spark] object

[GitHub] [spark] cloud-fan commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

cloud-fan commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656300953 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -377,8 +377,11 @@ class

[GitHub] [spark] AngersZhuuuu commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

AngersZh commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866053347 ping @MaxGekk @cloud-fan after check with PGSQL update the rule in the pr desc -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] otterc commented on a change in pull request #32140: [SPARK-32922][SHUFFLE][CORE] Adds support for executors to fetch local and remote merged shuffle data

otterc commented on a change in pull request #32140: URL: https://github.com/apache/spark/pull/32140#discussion_r656298069 ## File path: core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala ## @@ -0,0 +1,289 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] geekyouth closed pull request #33025: 基于 spark-2.4.3，合入了 spark3.0.0 的 from_json FAILFAST 功能，并且支持打印错误 jsonSchema 的字段名和字段值

geekyouth closed pull request #33025: URL: https://github.com/apache/spark/pull/33025 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] geekyouth opened a new pull request #33025: 基于 spark-2.4.3，合入了 spark3.0.0 的 from_json FAILFAST 功能，并且支持打印错误 jsonSchema 的字段名和字段值

geekyouth opened a new pull request #33025: URL: https://github.com/apache/spark/pull/33025 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

[GitHub] [spark] srowen commented on pull request #32973: Add missing GraphX classes to registerKryoClasses util method

srowen commented on pull request #32973: URL: https://github.com/apache/spark/pull/32973#issuecomment-866039864 (We'll want a JIRA for this too if it goes forward: https://spark.apache.org/contributing.html ) -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

AmplabJenkins removed a comment on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-864556548 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #32940: [SPARK-35768][SQL] Take into account year-month interval fields in cast

SparkQA commented on pull request #32940: URL: https://github.com/apache/spark/pull/32940#issuecomment-866035846 **[Test build #140147 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140147/testReport)** for PR 32940 at commit

[GitHub] [spark] SparkQA commented on pull request #32985: [SPARK-35777][SQL] Check all year-month interval types in UDF

SparkQA commented on pull request #32985: URL: https://github.com/apache/spark/pull/32985#issuecomment-866035666 **[Test build #140146 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140146/testReport)** for PR 32985 at commit

[GitHub] [spark] SparkQA commented on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

SparkQA commented on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866035744 **[Test build #140145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140145/testReport)** for PR 32987 at commit

[GitHub] [spark] SparkQA commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866035580 **[Test build #140144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140144/testReport)** for PR 33019 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins removed a comment on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866032361 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140143/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

AmplabJenkins removed a comment on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-866032360 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

AmplabJenkins removed a comment on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-866032357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44668/

[GitHub] [spark] AmplabJenkins commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866032361 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140143/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

AmplabJenkins commented on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-866032362 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] AmplabJenkins commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

AmplabJenkins commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-866032357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44668/ --

[GitHub] [spark] Ngone51 commented on a change in pull request #32868: [SPARK-35714][CORE] Bug fix for deadlock during the executor shutdown

Ngone51 commented on a change in pull request #32868: URL: https://github.com/apache/spark/pull/32868#discussion_r656275317 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/WorkerWatcher.scala ## @@ -45,7 +50,14 @@ private[spark] class WorkerWatcher(

[GitHub] [spark] AngersZhuuuu commented on pull request #32985: [SPARK-35777][SQL] Check all year-month interval types in UDF

AngersZh commented on pull request #32985: URL: https://github.com/apache/spark/pull/32985#issuecomment-866022502 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

SparkQA commented on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-866019833 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44669/ --

[GitHub] [spark] cloud-fan commented on a change in pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

cloud-fan commented on a change in pull request #32972: URL: https://github.com/apache/spark/pull/32972#discussion_r656259414 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -182,6 +182,15 @@ object ResolveUnion

[GitHub] [spark] cloud-fan commented on a change in pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

cloud-fan commented on a change in pull request #32972: URL: https://github.com/apache/spark/pull/32972#discussion_r656259828 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -182,6 +182,15 @@ object ResolveUnion

[GitHub] [spark] cloud-fan commented on a change in pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

cloud-fan commented on a change in pull request #32972: URL: https://github.com/apache/spark/pull/32972#discussion_r656259414 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala ## @@ -182,6 +182,15 @@ object ResolveUnion

[GitHub] [spark] cloud-fan commented on a change in pull request #33011: [SPARK-35841][SQL] Casting string to decimal type doesn't work if the…

cloud-fan commented on a change in pull request #33011: URL: https://github.com/apache/spark/pull/33011#discussion_r656256370 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/Decimal.scala ## @@ -587,12 +587,8 @@ object Decimal { } } - private

[GitHub] [spark] cloud-fan commented on pull request #32448: [SPARK-35290][SQL] Append new nested struct fields rather than sort for unionByName with null filling

cloud-fan commented on pull request #32448: URL: https://github.com/apache/spark/pull/32448#issuecomment-866011243 I'm +1 to open 2 fresh PRs, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA removed a comment on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

SparkQA removed a comment on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-865985217 **[Test build #140141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140141/testReport)** for PR 33023 at commit

[GitHub] [spark] SparkQA commented on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

SparkQA commented on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-866009215 **[Test build #140141 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140141/testReport)** for PR 33023 at commit

[GitHub] [spark] cloud-fan commented on pull request #32932: [SPARK-35786][SQL] Add a new operator to distingush if AQE can optimize safely

cloud-fan commented on pull request #32932: URL: https://github.com/apache/spark/pull/32932#issuecomment-866009262 The approach LGTM. My only concern is about the naming. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] gengliangwang commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

gengliangwang commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866008701 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] LuciferYang commented on pull request #32994: [SPARK-35838][BUILD][TESTS] Ensure all modules can be maven test independently in Scala 2.13

LuciferYang commented on pull request #32994: URL: https://github.com/apache/spark/pull/32994#issuecomment-866008664 thx ~ @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] tgravescs commented on pull request #32810: [SPARK-35672][CORE][YARN] Pass user classpath entries to executors using config instead of command line.

tgravescs commented on pull request #32810: URL: https://github.com/apache/spark/pull/32810#issuecomment-866008651 can you rekick tests, failures look unrelated -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on a change in pull request #32932: [SPARK-35786][SQL] Add a new operator to distingush if AQE can optimize safely

cloud-fan commented on a change in pull request #32932: URL: https://github.com/apache/spark/pull/32932#discussion_r656249153 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -1351,6 +1351,26 @@ object

[GitHub] [spark] tgravescs commented on a change in pull request #32804: [SPARK-26867][YARN] Spark Support of YARN Placement Constraint

tgravescs commented on a change in pull request #32804: URL: https://github.com/apache/spark/pull/32804#discussion_r656210527 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala ## @@ -27,6 +27,24 @@ import

[GitHub] [spark] tgravescs commented on pull request #32804: [SPARK-26867][YARN] Spark Support of YARN Placement Constraint

tgravescs commented on pull request #32804: URL: https://github.com/apache/spark/pull/32804#issuecomment-866006388 thanks for working on this, it looks very interesting > Add LocalityPreferredSchedulingRequestContainerPlacementStrategy for compute locality for SchedulingRequest

[GitHub] [spark] cloud-fan commented on a change in pull request #32932: [SPARK-35786][SQL] Add a new operator to distingush if AQE can optimize safely

cloud-fan commented on a change in pull request #32932: URL: https://github.com/apache/spark/pull/32932#discussion_r656246748 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveHints.scala ## @@ -196,11 +197,18 @@ object ResolveHints {

[GitHub] [spark] cloud-fan commented on a change in pull request #32932: [SPARK-35786][SQL] Add a new operator to distingush if AQE can optimize safely

cloud-fan commented on a change in pull request #32932: URL: https://github.com/apache/spark/pull/32932#discussion_r656246044 ## File path: docs/sql-performance-tuning.md ## @@ -228,6 +228,8 @@ The "REPARTITION_BY_RANGE" hint must have column names and a partition number is

[GitHub] [spark] SparkQA removed a comment on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA removed a comment on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865999089 **[Test build #140143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140143/testReport)** for PR 33019 at commit

[GitHub] [spark] SparkQA commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-866003734 **[Test build #140143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140143/testReport)** for PR 33019 at commit

[GitHub] [spark] yaooqinn commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

yaooqinn commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-866002542 > In any case, we can add a directional guide to the existing behavior. Let's not introducing another ignoring behavior. > > > Besides, we can not fail w/

[GitHub] [spark] cloud-fan commented on pull request #32987: [SPARK-35564][SQL] Support subexpression elimination for conditionally evaluated expressions

cloud-fan commented on pull request #32987: URL: https://github.com/apache/spark/pull/32987#issuecomment-866001630 OK to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

dongjoon-hyun edited a comment on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-866000318 In any case, we can add a directional guide to the existing behavior. Let's not introducing another ignoring behavior. > Besides, we can not fail w/

[GitHub] [spark] dongjoon-hyun commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

dongjoon-hyun commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-866000318 In any case, we can add a directional guide to the existing behavior. Not introducing another ignoring behavior. > Besides, we can not fail w/

[GitHub] [spark] SparkQA commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

SparkQA commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-865999585 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44668/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865999089 **[Test build #140143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140143/testReport)** for PR 33019 at commit

[GitHub] [spark] yaooqinn commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

yaooqinn commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865998245 Besides, we can not fail w/ `spark.kubernetes.driver.pod.name` and `client` mode directly. As you said, if users use that `.internal` config

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

dongjoon-hyun edited a comment on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865997248 Why do we need to follow YARN-specific behavior in our new K8s Spark UX if that is not better, @yaooqinn ? -- This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

dongjoon-hyun commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865997248 Why do we need to follow YARN-specific behavior in our new K8s Spark UX, @yaooqinn ? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan closed pull request #32999: [SPARK-35727][SQL] Return INTERVAL DAY from dates subtraction

cloud-fan closed pull request #32999: URL: https://github.com/apache/spark/pull/32999 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] cloud-fan commented on pull request #32999: [SPARK-35727][SQL] Return INTERVAL DAY from dates subtraction

cloud-fan commented on pull request #32999: URL: https://github.com/apache/spark/pull/32999#issuecomment-865996022 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] yaooqinn edited a comment on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

yaooqinn edited a comment on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865994837 how about unmanaged yarn application master with a lot of am settings？ -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] yaooqinn commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

yaooqinn commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865994837 how about unmanned yarn application master with a lot of am settings？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

SparkQA commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-865993046 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44668/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #32979: [SPARK-35828][K8S] Skip retrieving the non-exist driver pod for client mode

dongjoon-hyun commented on pull request #32979: URL: https://github.com/apache/spark/pull/32979#issuecomment-865992730 Do you think those are good Spark UX examples, @yaooqinn ? I don't think so. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins removed a comment on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865992342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140134/

[GitHub] [spark] AmplabJenkins commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

AmplabJenkins commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865992342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140134/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA removed a comment on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865833578 **[Test build #140134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140134/testReport)** for PR 33019 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33020: [WIP][SPARK-35543][CORE][FOLLOWUP] Fix memory leak in BlockManagerMasterEndpoint removeRdd

AmplabJenkins removed a comment on pull request #33020: URL: https://github.com/apache/spark/pull/33020#issuecomment-865990654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140138/

[GitHub] [spark] SparkQA commented on pull request #33019: [SPARK-35854][SQL] Improve the error message of to_timestamp_ntz with invalid format pattern

SparkQA commented on pull request #33019: URL: https://github.com/apache/spark/pull/33019#issuecomment-865991088 **[Test build #140134 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140134/testReport)** for PR 33019 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33020: [WIP][SPARK-35543][CORE][FOLLOWUP] Fix memory leak in BlockManagerMasterEndpoint removeRdd

AmplabJenkins commented on pull request #33020: URL: https://github.com/apache/spark/pull/33020#issuecomment-865990654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140138/ -- This

[GitHub] [spark] Peng-Lei commented on a change in pull request #32931: [SPARK-33898][SQL] Support SHOW CREATE TABLE In V2

Peng-Lei commented on a change in pull request #32931: URL: https://github.com/apache/spark/pull/32931#discussion_r656227832 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/ShowCreateTableExec.scala ## @@ -0,0 +1,148 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA removed a comment on pull request #33020: [WIP][SPARK-35543][CORE][FOLLOWUP] Fix memory leak in BlockManagerMasterEndpoint removeRdd

SparkQA removed a comment on pull request #33020: URL: https://github.com/apache/spark/pull/33020#issuecomment-865904517 **[Test build #140138 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140138/testReport)** for PR 33020 at commit

[GitHub] [spark] SparkQA commented on pull request #33020: [WIP][SPARK-35543][CORE][FOLLOWUP] Fix memory leak in BlockManagerMasterEndpoint removeRdd

SparkQA commented on pull request #33020: URL: https://github.com/apache/spark/pull/33020#issuecomment-865989040 **[Test build #140138 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140138/testReport)** for PR 33020 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

cloud-fan commented on a change in pull request #32850: URL: https://github.com/apache/spark/pull/32850#discussion_r656226073 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/interval.sql.out ## @@ -67,7 +67,7 @@ select interval '2 seconds' / 0 -- !query

[GitHub] [spark] dongjoon-hyun closed pull request #32994: [SPARK-35838][BUILD][TESTS] Ensure all modules can be maven test independently in Scala 2.13

dongjoon-hyun closed pull request #32994: URL: https://github.com/apache/spark/pull/32994 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] SparkQA commented on pull request #33022: [SPARK-35856][SQL][TESTS] Move new interval type test cases from CastSuite to CastBaseSuite

SparkQA commented on pull request #33022: URL: https://github.com/apache/spark/pull/33022#issuecomment-865985292 **[Test build #140142 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140142/testReport)** for PR 33022 at commit

[GitHub] [spark] SparkQA commented on pull request #33023: [SPARK-35812][PYTHON] Throw ValueError if version and timestamp are used together in to_delta

SparkQA commented on pull request #33023: URL: https://github.com/apache/spark/pull/33023#issuecomment-865985217 **[Test build #140141 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140141/testReport)** for PR 33023 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33024: [SPARK-35852][SQL] Use DateAdd instead of TimeAdd for DateType + INTERVAL DAY

AmplabJenkins commented on pull request #33024: URL: https://github.com/apache/spark/pull/33024#issuecomment-865984909 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on a change in pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

cloud-fan commented on a change in pull request #32850: URL: https://github.com/apache/spark/pull/32850#discussion_r656221248 ## File path: core/src/main/scala/org/apache/spark/SparkError.scala ## @@ -0,0 +1,85 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33021: [SPARK-35855][SQL] Unify reuse map data structures in non-AQE and AQE rules

AmplabJenkins removed a comment on pull request #33021: URL: https://github.com/apache/spark/pull/33021#issuecomment-865983629 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140137/

<    1   2   3   4   5   6   7   8   9   10   >