[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30245: [SPARK-33337][SQL] Support subexpression elimination in branches of conditional expressions

dongjoon-hyun commented on a change in pull request #30245: URL: https://github.com/apache/spark/pull/30245#discussion_r518408688 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -65,11 +65,46 @@ class

[GitHub] [spark] AmplabJenkins commented on pull request #30242: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

AmplabJenkins commented on pull request #30242: URL: https://github.com/apache/spark/pull/30242#issuecomment-722682936 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #30245: [SPARK-33337][SQL] Support subexpression elimination in branches of conditional expressions

dongjoon-hyun commented on a change in pull request #30245: URL: https://github.com/apache/spark/pull/30245#discussion_r518408118 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -65,11 +65,46 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

AmplabJenkins removed a comment on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722678961 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

SparkQA commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722678948 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35278/

[GitHub] [spark] AmplabJenkins commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

AmplabJenkins commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722678961 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30268: [SPARK-33365][BUILD] Update SBT to 1.4.2

SparkQA commented on pull request #30268: URL: https://github.com/apache/spark/pull/30268#issuecomment-722677810 **[Test build #130672 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130672/testReport)** for PR 30268 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

AmplabJenkins removed a comment on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722677553 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

AmplabJenkins commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722677553 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

SparkQA removed a comment on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722527733 **[Test build #130659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130659/testReport)** for PR 30264 at commit

[GitHub] [spark] venkata91 commented on a change in pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

venkata91 commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r518402471 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/scheduler/cluster/YarnSchedulerBackend.scala ## @@ -161,6 +170,27 @@

[GitHub] [spark] SparkQA commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

SparkQA commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722676685 **[Test build #130659 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130659/testReport)** for PR 30264 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28841: [SPARK-31962][SQL] Provide modifiedAfter and modifiedBefore options when filtering from a batch-based file data source

HeartSaVioR commented on pull request #28841: URL: https://github.com/apache/spark/pull/28841#issuecomment-722675458 I see you update the PR. Thanks! As you're upmerging the branch instead of rebasing it's uneasy to check the effective changes. Two questions: 1. Does your last

[GitHub] [spark] williamhyun opened a new pull request #30268: Update SBT to 1.4.2

williamhyun opened a new pull request #30268: URL: https://github.com/apache/spark/pull/30268 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How

[GitHub] [spark] SparkQA commented on pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

SparkQA commented on pull request #30154: URL: https://github.com/apache/spark/pull/30154#issuecomment-722674775 **[Test build #130670 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130670/testReport)** for PR 30154 at commit

[GitHub] [spark] SparkQA commented on pull request #30038: [SPARK-33130][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect)

SparkQA commented on pull request #30038: URL: https://github.com/apache/spark/pull/30038#issuecomment-722674810 **[Test build #130671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130671/testReport)** for PR 30038 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

AmplabJenkins removed a comment on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722673908 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] huaxingao commented on pull request #30038: [SPARK-33130][SQL] Support ALTER TABLE in JDBC v2 Table Catalog: add, update type and nullability of columns (MsSqlServer dialect)

huaxingao commented on pull request #30038: URL: https://github.com/apache/spark/pull/30038#issuecomment-722674427 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

AmplabJenkins removed a comment on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722673893 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

AmplabJenkins commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722673893 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

SparkQA commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722673607 **[Test build #130666 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130666/testReport)** for PR 30211 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

SparkQA removed a comment on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722620399 **[Test build #130666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130666/testReport)** for PR 30211 at commit

[GitHub] [spark] SparkQA commented on pull request #30242: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30242: URL: https://github.com/apache/spark/pull/30242#issuecomment-722673338 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35279/

[GitHub] [spark] huaxingao commented on a change in pull request #30154: [SPARK-32405][SQL] Apply table options while creating tables in JDBC Table Catalog

huaxingao commented on a change in pull request #30154: URL: https://github.com/apache/spark/pull/30154#discussion_r518397566 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/jdbc/JDBCTableCatalog.scala ## @@ -117,14 +117,33 @@ class

[GitHub] [spark] SparkQA commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

SparkQA commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722669412 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35278/

[GitHub] [spark] SparkQA commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

SparkQA commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722668185 **[Test build #130669 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130669/testReport)** for PR 30264 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

HeartSaVioR commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722667593 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HeartSaVioR commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

HeartSaVioR commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722667531 ``` Test build #130658 has finished for PR 30264 at commit d6e854c. This patch passes all tests. This patch does not merge cleanly. This patch adds no public

[GitHub] [spark] HeartSaVioR commented on pull request #30265: [SPARK-33362][SQL] skipSchemaResolution should still require query to be resolved

HeartSaVioR commented on pull request #30265: URL: https://github.com/apache/spark/pull/30265#issuecomment-722666513 Post LGTM. Thanks for the fix! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HeartSaVioR commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

HeartSaVioR commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-722662205 > BTW, shall we avoid traversing when maxChunksBeingTransferred=Long.MAX_VALUE? This is a good idea to optimize for the default value. Thanks!

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722659066 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722659058 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722659035 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35276/

[GitHub] [spark] AmplabJenkins commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722659058 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

AmplabJenkins removed a comment on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722658292 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

AmplabJenkins commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722658292 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

SparkQA commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722658273 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35277/

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

dongjoon-hyun commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r518381061 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientUtils.scala ## @@ -0,0 +1,124 @@

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

dongjoon-hyun commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r518379135 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientUtils.scala ## @@ -0,0 +1,128 @@

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

dongjoon-hyun commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r518379135 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/KubernetesClientUtils.scala ## @@ -0,0 +1,128 @@

[GitHub] [spark] SparkQA commented on pull request #30242: [SPARK-33277][PYSPARK][SQL] Use ContextAwareIterator to stop consuming after the task ends.

SparkQA commented on pull request #30242: URL: https://github.com/apache/spark/pull/30242#issuecomment-722654602 **[Test build #130668 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130668/testReport)** for PR 30242 at commit

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

dongjoon-hyun commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r518377620 ## File path: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackendSuite.scala

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #27735: [SPARK-30985][k8s] Support propagating SPARK_CONF_DIR files to driver and executor pods.

dongjoon-hyun commented on a change in pull request #27735: URL: https://github.com/apache/spark/pull/27735#discussion_r518376254 ## File path: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/deploy/k8s/features/BasicExecutorFeatureStepSuite.scala ## @@

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

AmplabJenkins removed a comment on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722649280 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

AmplabJenkins commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722649280 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722649257 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35275/

[GitHub] [spark] SparkQA commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

SparkQA commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722647065 **[Test build #130667 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130667/testReport)** for PR 30189 at commit

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722645553 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35276/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722644809 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] dongjoon-hyun commented on pull request #30255: [SPARK-33352][CORE] Fix procedure-like declaration compilation warnings of core module in Scala 2.13

dongjoon-hyun commented on pull request #30255: URL: https://github.com/apache/spark/pull/30255#issuecomment-722645272 +1 for @srowen 's [suggestion](https://github.com/apache/spark/pull/30255#issuecomment-722434912). This

[GitHub] [spark] SparkQA removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

SparkQA removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722620398 **[Test build #130665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130665/testReport)** for PR 30222 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins removed a comment on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722644802 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

AmplabJenkins commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722644802 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722644645 **[Test build #130665 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130665/testReport)** for PR 30222 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

dongjoon-hyun commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722644295 Retest this please. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #30189: [SPARK-33291][SQL] Improve DataFrame.show for nulls in arrays and structs

dongjoon-hyun commented on pull request #30189: URL: https://github.com/apache/spark/pull/30189#issuecomment-722644100 cc @viirya This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

SparkQA commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722643150 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35277/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30249: [SPARK-33342][WEBUI] fix the wrong url and display name of blocking thread in threadDump page.

AmplabJenkins removed a comment on pull request #30249: URL: https://github.com/apache/spark/pull/30249#issuecomment-722642991 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30249: [SPARK-33342][WEBUI] fix the wrong url and display name of blocking thread in threadDump page.

AmplabJenkins commented on pull request #30249: URL: https://github.com/apache/spark/pull/30249#issuecomment-722642991 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30249: [SPARK-33342][WEBUI] fix the wrong url and display name of blocking thread in threadDump page.

SparkQA removed a comment on pull request #30249: URL: https://github.com/apache/spark/pull/30249#issuecomment-722553485 **[Test build #130660 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130660/testReport)** for PR 30249 at commit

[GitHub] [spark] SparkQA commented on pull request #30249: [SPARK-33342][WEBUI] fix the wrong url and display name of blocking thread in threadDump page.

SparkQA commented on pull request #30249: URL: https://github.com/apache/spark/pull/30249#issuecomment-722641741 **[Test build #130660 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130660/testReport)** for PR 30249 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #30139: [SPARK-31069][CORE] high cpu caused by chunksBeingTransferred in external shuffle service

HeartSaVioR commented on pull request #30139: URL: https://github.com/apache/spark/pull/30139#issuecomment-722640952 > Another thought is, would it ease the problem if we track the transferring chunks per channel, e.g., Map[Channle, long]? I actually thought about this and looked

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722635027 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722635025 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722635002 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35274/

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722635025 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722634236 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35275/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722625730 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722625730 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722625710 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35273/

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722622832 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35274/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722622466 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722622435 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35272/

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722622466 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30211: [WIP][SPARK-33305][SQL] DSv2: DROP TABLE command should also invalidate cache

SparkQA commented on pull request #30211: URL: https://github.com/apache/spark/pull/30211#issuecomment-722620399 **[Test build #130666 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130666/testReport)** for PR 30211 at commit

[GitHub] [spark] SparkQA commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

SparkQA commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722620398 **[Test build #130665 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130665/testReport)** for PR 30222 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

AmplabJenkins removed a comment on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722617887 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

AmplabJenkins commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722617887 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #30222: [SPARK-33315][SQL] Simplify CaseWhen with EqualTo

dongjoon-hyun commented on pull request #30222: URL: https://github.com/apache/spark/pull/30222#issuecomment-722617489 Retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA removed a comment on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

SparkQA removed a comment on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722523372 **[Test build #130658 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130658/testReport)** for PR 30264 at commit

[GitHub] [spark] SparkQA commented on pull request #30264: [SPARK-33360][SQL] Simplify DS v2 write resolution

SparkQA commented on pull request #30264: URL: https://github.com/apache/spark/pull/30264#issuecomment-722616556 **[Test build #130658 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130658/testReport)** for PR 30264 at commit

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722614934 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35273/

[GitHub] [spark] SparkQA commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

SparkQA commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722613041 **[Test build #130664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130664/testReport)** for PR 30267 at commit

[GitHub] [spark] imback82 commented on pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

imback82 commented on pull request #30267: URL: https://github.com/apache/spark/pull/30267#issuecomment-722610985 cc @rdblue @cloud-fan This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] imback82 commented on a change in pull request #30079: [SPARK-33174][SQL] Migrate DROP TABLE to use UnresolvedTableOrView to resolve the identifier

imback82 commented on a change in pull request #30079: URL: https://github.com/apache/spark/pull/30079#discussion_r518329054 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -228,8 +228,11 @@ class

[GitHub] [spark] imback82 opened a new pull request #30267: [SPARK-33364][SQL] Introduce the "purge" option in TableCatalog.dropTable for v2 catalog.

imback82 opened a new pull request #30267: URL: https://github.com/apache/spark/pull/30267 ### What changes were proposed in this pull request? This PR proposes to introduce the `purge` option in `TableCatalog.dropTable` so that v2 catalogs can use the option if needed.

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722607751 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/35272/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722606479 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging par

SparkQA removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722601058 **[Test build #130663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130663/testReport)** for PR 30164 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722606469 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging parti

AmplabJenkins commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722606469 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722606448 **[Test build #130663 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130663/testReport)** for PR 30164 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30265: [SPARK-33362][SQL] skipSchemaResolution should still require query to be resolved

AmplabJenkins removed a comment on pull request #30265: URL: https://github.com/apache/spark/pull/30265#issuecomment-722601104 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722601058 **[Test build #130663 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130663/testReport)** for PR 30164 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #30265: [SPARK-33362][SQL] skipSchemaResolution should still require query to be resolved

AmplabJenkins commented on pull request #30265: URL: https://github.com/apache/spark/pull/30265#issuecomment-722601104 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #30265: [SPARK-33362][SQL] skipSchemaResolution should still require query to be resolved

SparkQA removed a comment on pull request #30265: URL: https://github.com/apache/spark/pull/30265#issuecomment-722434986 **[Test build #130654 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130654/testReport)** for PR 30265 at commit

[GitHub] [spark] SparkQA commented on pull request #30265: [SPARK-33362][SQL] skipSchemaResolution should still require query to be resolved

SparkQA commented on pull request #30265: URL: https://github.com/apache/spark/pull/30265#issuecomment-722599699 **[Test build #130654 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130654/testReport)** for PR 30265 at commit

[GitHub] [spark] venkata91 commented on a change in pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

venkata91 commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r518315690 ## File path: core/src/main/scala/org/apache/spark/Dependency.scala ## @@ -95,6 +97,30 @@ class ShuffleDependency[K: ClassTag, V: ClassTag, C:

[GitHub] [spark] venkata91 commented on a change in pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

venkata91 commented on a change in pull request #30164: URL: https://github.com/apache/spark/pull/30164#discussion_r518314381 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1927,4 +1927,34 @@ package object config {

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-722591957 Test FAILed. Refer to this link for build results (access rights to CI server needed):

<    1   2   3   4   5   6   7   8   >