[GitHub] [spark] SparkQA commented on pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

SparkQA commented on pull request #33141: URL: https://github.com/apache/spark/pull/33141#issuecomment-871769117 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44977/ -- This is an automated message from the Apache

[GitHub] [spark] otterc commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

otterc commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661812478 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/FetchShuffleBlocks.java ## @@ -42,10 +42,11 @@ public FetchSh

[GitHub] [spark] SparkQA commented on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

SparkQA commented on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871764190 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44976/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

SparkQA removed a comment on pull request #33141: URL: https://github.com/apache/spark/pull/33141#issuecomment-871750546 **[Test build #140463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140463/testReport)** for PR 33141 at commit [`bd037d4`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

SparkQA commented on pull request #33141: URL: https://github.com/apache/spark/pull/33141#issuecomment-871763696 **[Test build #140463 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140463/testReport)** for PR 33141 at commit [`bd037d4`](https://github.co

[GitHub] [spark] xinrong-databricks closed pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

xinrong-databricks closed pull request #33141: URL: https://github.com/apache/spark/pull/33141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: rev

[GitHub] [spark] zsxwing commented on a change in pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

zsxwing commented on a change in pull request #33093: URL: https://github.com/apache/spark/pull/33093#discussion_r661806677 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/UnsupportedOperationChecker.scala ## @@ -232,6 +232,10 @@ object Unsuppo

[GitHub] [spark] SparkQA commented on pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

SparkQA commented on pull request #33141: URL: https://github.com/apache/spark/pull/33141#issuecomment-871750546 **[Test build #140463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140463/testReport)** for PR 33141 at commit [`bd037d4`](https://github.com

[GitHub] [spark] xinrong-databricks commented on a change in pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

xinrong-databricks commented on a change in pull request #33141: URL: https://github.com/apache/spark/pull/33141#discussion_r661833111 ## File path: docs/rdd-programming-guide.md ## @@ -105,6 +105,7 @@ Spark {{site.SPARK_VERSION}} works with Python 3.6+. It can use the standar

[GitHub] [spark] srowen commented on pull request #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade

srowen commented on pull request #27370: URL: https://github.com/apache/spark/pull/27370#issuecomment-871748460 The commit message above shows the commit it produces, and that it's in branch 3.1. The JIRA says 3.1.0 https://github.com/apache/spark/commit/2a4fed0443d6fe066219124833782293

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

AmplabJenkins removed a comment on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871743469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

AmplabJenkins removed a comment on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871743470 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140462/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871743472 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140458/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

AmplabJenkins removed a comment on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871743468 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140454/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

AmplabJenkins commented on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871743470 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140462/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

AmplabJenkins commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871743469 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871743472 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140458/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

AmplabJenkins commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871743468 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140454/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

SparkQA removed a comment on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871721289 **[Test build #140462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140462/testReport)** for PR 33139 at commit [`531972d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

SparkQA commented on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871737133 **[Test build #140462 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140462/testReport)** for PR 33139 at commit [`531972d`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

dongjoon-hyun commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871737030 Also, cc @gengliangwang since this was the release blocker for Apache Spark 3.2.0. -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] dongjoon-hyun closed pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

dongjoon-hyun closed pull request #32753: URL: https://github.com/apache/spark/pull/32753 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] dongjoon-hyun commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

dongjoon-hyun commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871736077 Merged to master for Apache Spark 3.2.0. Thank you, @sunchao , @viirya , @cloud-fan cc @aokolnychyi , @RussellSpitzer -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871639364 **[Test build #140458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140458/testReport)** for PR 33093 at commit [`a25ea47`](https://gi

[GitHub] [spark] emmanuelrflores commented on pull request #27370: [SPARK-30654][WebUI] Bootstrap4 WebUI upgrade

emmanuelrflores commented on pull request #27370: URL: https://github.com/apache/spark/pull/27370#issuecomment-871734336 Has this been put into a release? Looking at [commit](https://github.com/apache/spark/commit/2a4fed0443d6fe066219124833782293630f8a89) by @gengliangwang, it looks like i

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871734168 **[Test build #140458 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140458/testReport)** for PR 33093 at commit [`a25ea47`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871729767 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44975/ -- This is an automated message from the A

[GitHub] [spark] ueshin commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

ueshin commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871729051 cc @HyukjinKwon @itholic @xinrong-databricks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] otterc commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

otterc commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661811779 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/protocol/AbstractFetchShuffleBlocks.java ## @@ -33,21 +33,25 @@ pub

[GitHub] [spark] SparkQA removed a comment on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA removed a comment on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871710680 **[Test build #140461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140461/testReport)** for PR 33159 at commit [`4e6bf88`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871728095 **[Test build #140461 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140461/testReport)** for PR 33159 at commit [`4e6bf88`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

SparkQA removed a comment on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871561948 **[Test build #140454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140454/testReport)** for PR 32933 at commit [`20db6b5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

SparkQA commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871725113 **[Test build #140454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140454/testReport)** for PR 32933 at commit [`20db6b5`](https://github.co

[GitHub] [spark] xinrong-databricks commented on a change in pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

xinrong-databricks commented on a change in pull request #33139: URL: https://github.com/apache/spark/pull/33139#discussion_r661805155 ## File path: python/pyspark/context.py ## @@ -230,6 +230,14 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize

[GitHub] [spark] SparkQA commented on pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

SparkQA commented on pull request #33139: URL: https://github.com/apache/spark/pull/33139#issuecomment-871721289 **[Test build #140462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140462/testReport)** for PR 33139 at commit [`531972d`](https://github.com

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661801577 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661799051 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661799051 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661801577 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661178449 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661799051 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] ueshin commented on a change in pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

ueshin commented on a change in pull request #33139: URL: https://github.com/apache/spark/pull/33139#discussion_r661796890 ## File path: python/pyspark/context.py ## @@ -230,6 +230,14 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, s

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871710680 **[Test build #140461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140461/testReport)** for PR 33159 at commit [`4e6bf88`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33158: [SPARK-35888][SQL][FOLLOWUP] Return partition specs for all the shuffles

AmplabJenkins removed a comment on pull request #33158: URL: https://github.com/apache/spark/pull/33158#issuecomment-871710091 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44971/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

AmplabJenkins removed a comment on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871710087 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44973/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

AmplabJenkins removed a comment on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871710085 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

AmplabJenkins removed a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-871710089 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140456/ -

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871710152 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44975/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

AmplabJenkins commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871710087 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44973/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

AmplabJenkins commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-871710089 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140456/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33158: [SPARK-35888][SQL][FOLLOWUP] Return partition specs for all the shuffles

AmplabJenkins commented on pull request #33158: URL: https://github.com/apache/spark/pull/33158#issuecomment-871710091 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44971/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

AmplabJenkins commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871710086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] Victsm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

Victsm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661791974 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlockM

[GitHub] [spark] mridulm commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state

mridulm commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r661786876 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -403,38 +394,78 @@ public MergeSta

[GitHub] [spark] mridulm commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state

mridulm commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r661786876 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -403,38 +394,78 @@ public MergeSta

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871703539 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44974/ -- This

[GitHub] [spark] SparkQA commented on pull request #33158: [SPARK-35888][SQL][FOLLOWUP] Return partition specs for all the shuffles

SparkQA commented on pull request #33158: URL: https://github.com/apache/spark/pull/33158#issuecomment-871696884 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44971/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

SparkQA removed a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-871601512 **[Test build #140456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140456/testReport)** for PR 33101 at commit [`983bb36`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

SparkQA commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-871696605 **[Test build #140456 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140456/testReport)** for PR 33101 at commit [`983bb36`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

SparkQA commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871693648 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44973/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA removed a comment on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871675002 **[Test build #140460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140460/testReport)** for PR 33159 at commit [`979fa48`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871692027 **[Test build #140460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140460/testReport)** for PR 33159 at commit [`979fa48`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA removed a comment on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871509209 **[Test build #140449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140449/testReport)** for PR 33140 at commit [`9e52fa0`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871680630 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140449/ -- This

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871679464 **[Test build #140449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140449/testReport)** for PR 33140 at commit [`9e52fa0`](https://github.co

[GitHub] [spark] aokolnychyi commented on a change in pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

aokolnychyi commented on a change in pull request #33008: URL: https://github.com/apache/spark/pull/33008#discussion_r661761623 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsRowLevelOperations.java ## @@ -0,0 +1,43 @@ +/* + * Licensed

[GitHub] [spark] SparkQA commented on pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

SparkQA commented on pull request #33159: URL: https://github.com/apache/spark/pull/33159#issuecomment-871675002 **[Test build #140460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140460/testReport)** for PR 33159 at commit [`979fa48`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32831: [SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

AmplabJenkins removed a comment on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-871673829 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44970/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871673830 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44972/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33156: [SPARK-35953][SQL] Support extracting date fields from timestamp without time zone

AmplabJenkins removed a comment on pull request #33156: URL: https://github.com/apache/spark/pull/33156#issuecomment-871673828 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140440/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

AmplabJenkins removed a comment on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871673834 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140444/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

AmplabJenkins removed a comment on pull request #32972: URL: https://github.com/apache/spark/pull/32972#issuecomment-871673832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140443/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33155: [SPARK-35950][WebUI] Failed to toggle Exec Loss Reason in the executors page

AmplabJenkins removed a comment on pull request #33155: URL: https://github.com/apache/spark/pull/33155#issuecomment-871673827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140452/ -

[GitHub] [spark] SparkQA commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

SparkQA commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871674095 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44973/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33158: [SPARK-35888][SQL][FOLLOWUP] Return partition specs for all the shuffles

SparkQA commented on pull request #33158: URL: https://github.com/apache/spark/pull/33158#issuecomment-871673996 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44971/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33155: [SPARK-35950][WebUI] Failed to toggle Exec Loss Reason in the executors page

AmplabJenkins commented on pull request #33155: URL: https://github.com/apache/spark/pull/33155#issuecomment-871673827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140452/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33156: [SPARK-35953][SQL] Support extracting date fields from timestamp without time zone

AmplabJenkins commented on pull request #33156: URL: https://github.com/apache/spark/pull/33156#issuecomment-871673828 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140440/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

AmplabJenkins commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871673834 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140444/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32831: [SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

AmplabJenkins commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-871673829 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44970/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

AmplabJenkins commented on pull request #32972: URL: https://github.com/apache/spark/pull/32972#issuecomment-871673832 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140443/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871673830 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44972/ -- T

[GitHub] [spark] aokolnychyi commented on a change in pull request #33008: [WIP][SPARK-35801][SQL] Support DELETE operations that require rewriting data

aokolnychyi commented on a change in pull request #33008: URL: https://github.com/apache/spark/pull/33008#discussion_r661753646 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/SupportsRowLevelOperations.java ## @@ -0,0 +1,43 @@ +/* + * Licensed

[GitHub] [spark] ueshin opened a new pull request #33159: [SPARK-35944][PYTHON] Introduce Name and Label type aliases

ueshin opened a new pull request #33159: URL: https://github.com/apache/spark/pull/33159 ### What changes were proposed in this pull request? Introduce `Name` and `Label` type aliases to distinguish what is expected instead of `Union[Any, Tuple]`. - `Label`: `Tuple[Any, ...]`

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871665465 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44972/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

SparkQA removed a comment on pull request #32972: URL: https://github.com/apache/spark/pull/32972#issuecomment-871461540 **[Test build #140443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140443/testReport)** for PR 32972 at commit [`a29df92`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32972: [SPARK-35756][SQL] unionByName supports struct having same col names but different sequence

SparkQA commented on pull request #32972: URL: https://github.com/apache/spark/pull/32972#issuecomment-871659836 **[Test build #140443 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140443/testReport)** for PR 32972 at commit [`a29df92`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33155: [SPARK-35950][WebUI] Failed to toggle Exec Loss Reason in the executors page

SparkQA removed a comment on pull request #33155: URL: https://github.com/apache/spark/pull/33155#issuecomment-871561722 **[Test build #140452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140452/testReport)** for PR 33155 at commit [`c3e058e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33155: [SPARK-35950][WebUI] Failed to toggle Exec Loss Reason in the executors page

SparkQA commented on pull request #33155: URL: https://github.com/apache/spark/pull/33155#issuecomment-871657567 **[Test build #140452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140452/testReport)** for PR 33155 at commit [`c3e058e`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

SparkQA removed a comment on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871461502 **[Test build #140444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140444/testReport)** for PR 32933 at commit [`368ae75`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33156: [SPARK-35953][SQL] Support extracting date fields from timestamp without time zone

SparkQA removed a comment on pull request #33156: URL: https://github.com/apache/spark/pull/33156#issuecomment-871461759 **[Test build #140440 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140440/testReport)** for PR 33156 at commit [`afa6903`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

SparkQA commented on pull request #32933: URL: https://github.com/apache/spark/pull/32933#issuecomment-871652616 **[Test build #140444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140444/testReport)** for PR 32933 at commit [`368ae75`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32831: [SPARK-35685][SQL] Prompt recreating the view when there is an incompatible schema issue

SparkQA commented on pull request #32831: URL: https://github.com/apache/spark/pull/32831#issuecomment-871652256 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44970/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33156: [SPARK-35953][SQL] Support extracting date fields from timestamp without time zone

SparkQA commented on pull request #33156: URL: https://github.com/apache/spark/pull/33156#issuecomment-871651649 **[Test build #140440 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140440/testReport)** for PR 33156 at commit [`afa6903`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

SparkQA commented on pull request #32753: URL: https://github.com/apache/spark/pull/32753#issuecomment-871646642 **[Test build #140459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140459/testReport)** for PR 32753 at commit [`6541d99`](https://github.com

[GitHub] [spark] ueshin commented on a change in pull request #33141: [WIP][SPARK-35939][DOCS][PYTHON] Deprecate Python 3.6 in Spark documentation

ueshin commented on a change in pull request #33141: URL: https://github.com/apache/spark/pull/33141#discussion_r661729443 ## File path: docs/rdd-programming-guide.md ## @@ -105,6 +105,7 @@ Spark {{site.SPARK_VERSION}} works with Python 3.6+. It can use the standard CPy so C

[GitHub] [spark] ueshin commented on a change in pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

ueshin commented on a change in pull request #33139: URL: https://github.com/apache/spark/pull/33139#discussion_r661727449 ## File path: python/pyspark/context.py ## @@ -230,6 +230,14 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, s

[GitHub] [spark] ueshin commented on a change in pull request #33139: [SPARK-35938][PYTHON] Add deprecation warning for Python 3.6

ueshin commented on a change in pull request #33139: URL: https://github.com/apache/spark/pull/33139#discussion_r661727003 ## File path: python/pyspark/context.py ## @@ -230,6 +230,14 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize, s

[GitHub] [spark] AmplabJenkins commented on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

AmplabJenkins commented on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-871637494 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871641279 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] sunchao commented on a change in pull request #32753: [SPARK-34859][SQL] Handle column index when using vectorized Parquet reader

sunchao commented on a change in pull request #32753: URL: https://github.com/apache/spark/pull/32753#discussion_r661724331 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedRleValuesReader.java ## @@ -156,55 +156,81 @@ public in

<    1   2   3   4   5   6   7   8   >