[GitHub] [spark] AmplabJenkins commented on pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

AmplabJenkins commented on pull request #32161: URL: https://github.com/apache/spark/pull/32161#issuecomment-845709352 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43303/ -- T

[GitHub] [spark] SparkQA commented on pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

SparkQA commented on pull request #32513: URL: https://github.com/apache/spark/pull/32513#issuecomment-845709221 **[Test build #138794 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138794/testReport)** for PR 32513 at commit [`fde3c31`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32590: [SPARK-35445][SQL] Reduce the execution time of DeduplicateRelations

AmplabJenkins commented on pull request #32590: URL: https://github.com/apache/spark/pull/32590#issuecomment-845708887 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43299/ -- T

[GitHub] [spark] SparkQA commented on pull request #32606: [SPARK-35287][SQL] Allow RemoveRedundantProjects to preserve ProjectExec which generates UnsafeRow for DataSourceV2ScanRelation

SparkQA commented on pull request #32606: URL: https://github.com/apache/spark/pull/32606#issuecomment-845709067 **[Test build #138792 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138792/testReport)** for PR 32606 at commit [`c2f5f8f`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32606: [SPARK-35287][SQL] Allow RemoveRedundantProjects to preserve ProjectExec which generates UnsafeRow for DataSourceV2ScanRelation

AmplabJenkins commented on pull request #32606: URL: https://github.com/apache/spark/pull/32606#issuecomment-845708827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43298/ -- T

[GitHub] [spark] SparkQA commented on pull request #32602: [SPARK-35455][SQL] Enhance EliminateUnnecessaryJoin

SparkQA commented on pull request #32602: URL: https://github.com/apache/spark/pull/32602#issuecomment-845709077 **[Test build #138793 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138793/testReport)** for PR 32602 at commit [`7b80db0`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32615: [SPARK-35479][SQL] Format PartitionFilters IN strings in scan nodes

AmplabJenkins commented on pull request #32615: URL: https://github.com/apache/spark/pull/32615#issuecomment-845708849 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43297/ -- T

[GitHub] [spark] sarutak commented on a change in pull request #32606: [SPARK-35287][SQL] Allow RemoveRedundantProjects to preserve ProjectExec which generates UnsafeRow for DataSourceV2ScanRelation

sarutak commented on a change in pull request #32606: URL: https://github.com/apache/spark/pull/32606#discussion_r636677458 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/RemoveRedundantProjectsSuite.scala ## @@ -215,6 +217,27 @@ abstract class RemoveRedu

[GitHub] [spark] AmplabJenkins commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

AmplabJenkins commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845707438 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43309/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

AmplabJenkins commented on pull request #32610: URL: https://github.com/apache/spark/pull/32610#issuecomment-845707440 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138791/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845707439 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43311/ -- T

[GitHub] [spark] itholic edited a comment on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

itholic edited a comment on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845706476 Thanks, @HyukjinKwon . PR description is updated, and also the PR description of https://github.com/apache/spark/pull/32204, https://github.com/apache/spark/pull/32161

[GitHub] [spark] itholic commented on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

itholic commented on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845706476 Thanks, @HyukjinKwon . PR description is updated, and also https://github.com/apache/spark/pull/32204, https://github.com/apache/spark/pull/32161 are updated as well. -- T

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845702336 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43311/ -- This

[GitHub] [spark] maropu commented on pull request #32615: [SPARK-35479][SQL] Format PartitionFilters IN strings in scan nodes

maropu commented on pull request #32615: URL: https://github.com/apache/spark/pull/32615#issuecomment-845699926 Thank you, @cloud-fan @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] yaooqinn commented on a change in pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

yaooqinn commented on a change in pull request #32610: URL: https://github.com/apache/spark/pull/32610#discussion_r636665032 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -250,11 +250,21 @@ private[spark] object C

[GitHub] [spark] yaooqinn commented on a change in pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

yaooqinn commented on a change in pull request #32610: URL: https://github.com/apache/spark/pull/32610#discussion_r636665032 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -250,11 +250,21 @@ private[spark] object C

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r636671107 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/CommandResult.scala ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r636670859 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/CommandResult.scala ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r63865 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [spark] Ngone51 commented on pull request #32136: [SPARK-35022][CORE] Task Scheduling Plugin in Spark

Ngone51 commented on pull request #32136: URL: https://github.com/apache/spark/pull/32136#issuecomment-845694478 > For the stage level scheduling option, is the state store essentially the same across all executors? No. Tasks with the different partition ids must use the different st

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r636668841 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -74,12 +75,26 @@ class QueryExecution( sparkSessio

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r636668521 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ## @@ -74,12 +75,26 @@ class QueryExecution( sparkSessio

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r63865 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [spark] cloud-fan commented on a change in pull request #32513: [SPARK-35378][SQL] Eagerly execute non-root Command so that query command with CTE

cloud-fan commented on a change in pull request #32513: URL: https://github.com/apache/spark/pull/32513#discussion_r63474 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/CommandResultExec.scala ## @@ -0,0 +1,93 @@ +/* + * Licensed to the Apache Softwar

[GitHub] [spark] HeartSaVioR commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

HeartSaVioR commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845690788 cc. @viirya @gaborgsomogyi @xuanyuanking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[GitHub] [spark] SparkQA removed a comment on pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

SparkQA removed a comment on pull request #32610: URL: https://github.com/apache/spark/pull/32610#issuecomment-845682707 **[Test build #138791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138791/testReport)** for PR 32610 at commit [`c04dc4c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

SparkQA commented on pull request #32610: URL: https://github.com/apache/spark/pull/32610#issuecomment-845690497 **[Test build #138791 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138791/testReport)** for PR 32610 at commit [`c04dc4c`](https://github.co

[GitHub] [spark] yaooqinn commented on a change in pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

yaooqinn commented on a change in pull request #32610: URL: https://github.com/apache/spark/pull/32610#discussion_r636665032 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -250,11 +250,21 @@ private[spark] object C

[GitHub] [spark] SparkQA commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

SparkQA commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845688692 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43309/ -- This is an automated message from the A

[GitHub] [spark] beliefer commented on pull request #32478: [SPARK-35063][SQL] Group exception messages in sql/catalyst

beliefer commented on pull request #32478: URL: https://github.com/apache/spark/pull/32478#issuecomment-845687806 @allisonwang-db Thanks you for your review @cloud-fan Thank you too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on t

[GitHub] [spark] cloud-fan closed pull request #32574: [SPARK-35427][SQL][TESTS] Check the `EXCEPTION` rebase mode for Avro/Parquet

cloud-fan closed pull request #32574: URL: https://github.com/apache/spark/pull/32574 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [spark] cloud-fan commented on pull request #32574: [SPARK-35427][SQL][TESTS] Check the `EXCEPTION` rebase mode for Avro/Parquet

cloud-fan commented on pull request #32574: URL: https://github.com/apache/spark/pull/32574#issuecomment-845686175 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #32478: [SPARK-35063][SQL] Group exception messages in sql/catalyst

cloud-fan closed pull request #32478: URL: https://github.com/apache/spark/pull/32478 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [spark] cloud-fan commented on pull request #32478: [SPARK-35063][SQL] Group exception messages in sql/catalyst

cloud-fan commented on pull request #32478: URL: https://github.com/apache/spark/pull/32478#issuecomment-845684832 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] Ngone51 commented on a change in pull request #32616: [SPARK-35454][SQL] One LogicalPlan can match multiple dataset ids

Ngone51 commented on a change in pull request #32616: URL: https://github.com/apache/spark/pull/32616#discussion_r636660566 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -231,9 +231,10 @@ class Dataset[T] private[sql]( case _ =>

[GitHub] [spark] SparkQA commented on pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

SparkQA commented on pull request #32610: URL: https://github.com/apache/spark/pull/32610#issuecomment-845682707 **[Test build #138791 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138791/testReport)** for PR 32610 at commit [`c04dc4c`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

AmplabJenkins removed a comment on pull request #32161: URL: https://github.com/apache/spark/pull/32161#issuecomment-845675575 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43310/

[GitHub] [spark] SparkQA commented on pull request #32587: [SPARK-35440][SQL] Add function type to `ExpressionInfo` for UDF

SparkQA commented on pull request #32587: URL: https://github.com/apache/spark/pull/32587#issuecomment-845677285 **[Test build #138790 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138790/testReport)** for PR 32587 at commit [`69236de`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

AmplabJenkins commented on pull request #32161: URL: https://github.com/apache/spark/pull/32161#issuecomment-845675575 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43310/ -- T

[GitHub] [spark] SparkQA commented on pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

SparkQA commented on pull request #32161: URL: https://github.com/apache/spark/pull/32161#issuecomment-845672347 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43310/ -- This

[GitHub] [spark] SparkQA commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

SparkQA commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845671780 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43309/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan closed pull request #32615: [SPARK-35479][SQL] Format PartitionFilters IN strings in scan nodes

cloud-fan closed pull request #32615: URL: https://github.com/apache/spark/pull/32615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [spark] cloud-fan commented on pull request #32615: [SPARK-35479][SQL] Format PartitionFilters IN strings in scan nodes

cloud-fan commented on pull request #32615: URL: https://github.com/apache/spark/pull/32615#issuecomment-845671031 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] Ngone51 commented on pull request #32590: [SPARK-35445][SQL] Reduce the execution time of DeduplicateRelations

Ngone51 commented on pull request #32590: URL: https://github.com/apache/spark/pull/32590#issuecomment-845665122 thanks all! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] gengliangwang closed pull request #32590: [SPARK-35445][SQL] Reduce the execution time of DeduplicateRelations

gengliangwang closed pull request #32590: URL: https://github.com/apache/spark/pull/32590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] gengliangwang commented on pull request #32590: [SPARK-35445][SQL] Reduce the execution time of DeduplicateRelations

gengliangwang commented on pull request #32590: URL: https://github.com/apache/spark/pull/32590#issuecomment-845662291 Thanks, merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[GitHub] [spark] HyukjinKwon closed pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

HyukjinKwon closed pull request #32600: URL: https://github.com/apache/spark/pull/32600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] HyukjinKwon commented on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

HyukjinKwon commented on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845661750 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] AmplabJenkins commented on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

AmplabJenkins commented on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845658495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138781/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

AmplabJenkins removed a comment on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845658495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138781/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

SparkQA removed a comment on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845597607 **[Test build #138781 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138781/testReport)** for PR 32600 at commit [`d41a1be`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

SparkQA commented on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845657587 **[Test build #138781 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138781/testReport)** for PR 32600 at commit [`d41a1be`](https://github.co

[GitHub] [spark] MaxGekk commented on pull request #32574: [SPARK-35427][SQL][TESTS] Check the `EXCEPTION` rebase mode for Avro/Parquet

MaxGekk commented on pull request #32574: URL: https://github.com/apache/spark/pull/32574#issuecomment-845655136 @gengliangwang @HyukjinKwon Could you take a look at the PR, please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [spark] allisonwang-db commented on a change in pull request #32606: [SPARK-35287][SQL] Allow RemoveRedundantProjects to preserve ProjectExec which generates UnsafeRow for DataSourceV2ScanRel

allisonwang-db commented on a change in pull request #32606: URL: https://github.com/apache/spark/pull/32606#discussion_r636635430 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/RemoveRedundantProjectsSuite.scala ## @@ -215,6 +217,27 @@ abstract class Rem

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

AmplabJenkins removed a comment on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-845652568 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43304/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins removed a comment on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845652569 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43308/

[GitHub] [spark] AmplabJenkins commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845652569 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43308/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

AmplabJenkins commented on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-845652568 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43304/ -- T

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845652182 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43308/ -- This

[GitHub] [spark] MaxGekk commented on pull request #32574: [SPARK-35427][SQL][TESTS] Check the `EXCEPTION` rebase mode for Avro/Parquet

MaxGekk commented on pull request #32574: URL: https://github.com/apache/spark/pull/32574#issuecomment-845649011 @cloud-fan Any objections to the changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

SparkQA commented on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-845645932 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43304/ -- This is an automated message from the A

[GitHub] [spark] itholic commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

itholic commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636625094 ## File path: docs/sql-data-sources-orc.md ## @@ -172,3 +172,29 @@ When reading from Hive metastore ORC tables and inserting to Hive metastore ORC 2.0

[GitHub] [spark] chrismbryant commented on pull request #27278: [SPARK-30569][SQL][PYSPARK][SPARKR] Add percentile_approx DSL functions.

chrismbryant commented on pull request #27278: URL: https://github.com/apache/spark/pull/27278#issuecomment-845642088 @HyukjinKwon Thanks, here's that ticket: https://issues.apache.org/jira/browse/SPARK-35480 -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #32577: [SPARK-35422][SQL] Fix plan-printing issues to pass the TPCDS plan stability tests in Scala v2.13

HyukjinKwon commented on pull request #32577: URL: https://github.com/apache/spark/pull/32577#issuecomment-845639671 Nice, LGTM2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [spark] yaooqinn commented on a change in pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

yaooqinn commented on a change in pull request #32600: URL: https://github.com/apache/spark/pull/32600#discussion_r636621258 ## File path: core/src/main/scala/org/apache/spark/internal/config/ConfigBuilder.scala ## @@ -104,7 +104,7 @@ private[spark] class TypedConfigBuilder[T]

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

AmplabJenkins removed a comment on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845638217 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43305/

[GitHub] [spark] AmplabJenkins commented on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

AmplabJenkins commented on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845638217 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43305/ -- T

[GitHub] [spark] SparkQA commented on pull request #32600: [SPARK-35456][CORE] Print the invalid value in config validation error message

SparkQA commented on pull request #32600: URL: https://github.com/apache/spark/pull/32600#issuecomment-845638203 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43305/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32586: [SPARK-35439][SQL] Children subexpr should come first than parent subexpr

AmplabJenkins removed a comment on pull request #32586: URL: https://github.com/apache/spark/pull/32586#issuecomment-845634701 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138770/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32586: [SPARK-35439][SQL] Children subexpr should come first than parent subexpr

SparkQA removed a comment on pull request #32586: URL: https://github.com/apache/spark/pull/32586#issuecomment-845535109 **[Test build #138770 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138770/testReport)** for PR 32586 at commit [`3819bf3`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins removed a comment on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845635353 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138788/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

AmplabJenkins removed a comment on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845634865 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138786/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA removed a comment on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845635027 **[Test build #138788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138788/testReport)** for PR 32611 at commit [`e626c52`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32204: [SPARK-34494][SQL][DOCS] Move JSON data source options from Python and Scala into a single page

AmplabJenkins removed a comment on pull request #32204: URL: https://github.com/apache/spark/pull/32204#issuecomment-845634702 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43302/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

AmplabJenkins removed a comment on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845634703 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43300/

[GitHub] [spark] SparkQA removed a comment on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

SparkQA removed a comment on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845619416 **[Test build #138786 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138786/testReport)** for PR 32609 at commit [`ec1f662`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845635353 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138788/ -- This

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845635337 **[Test build #138788 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138788/testReport)** for PR 32611 at commit [`e626c52`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32587: [SPARK-35440][SQL] Add function type to `ExpressionInfo` for UDF

SparkQA commented on pull request #32587: URL: https://github.com/apache/spark/pull/32587#issuecomment-845635076 **[Test build #138789 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138789/testReport)** for PR 32587 at commit [`f7d01f1`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31830: [SPARK-34735][SQL][UI] Add modified configs for SQL execution in UI

SparkQA commented on pull request #31830: URL: https://github.com/apache/spark/pull/31830#issuecomment-845635070 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43304/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-845635027 **[Test build #138788 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138788/testReport)** for PR 32611 at commit [`e626c52`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

AmplabJenkins commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845634865 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138786/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32204: [SPARK-34494][SQL][DOCS] Move JSON data source options from Python and Scala into a single page

AmplabJenkins commented on pull request #32204: URL: https://github.com/apache/spark/pull/32204#issuecomment-845634702 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43302/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32586: [SPARK-35439][SQL] Children subexpr should come first than parent subexpr

AmplabJenkins commented on pull request #32586: URL: https://github.com/apache/spark/pull/32586#issuecomment-845634701 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138770/ -- This

[GitHub] [spark] SparkQA commented on pull request #32609: [SPARK-29223][SQL][SS] New option to specify timestamp on all subscribing topic-partitions in Kafka source

SparkQA commented on pull request #32609: URL: https://github.com/apache/spark/pull/32609#issuecomment-845634723 **[Test build #138786 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138786/testReport)** for PR 32609 at commit [`ec1f662`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

AmplabJenkins commented on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845634703 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43300/ -- T

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

dongjoon-hyun commented on a change in pull request #32610: URL: https://github.com/apache/spark/pull/32610#discussion_r636616275 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -250,11 +250,21 @@ private[spark] obj

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun edited a comment on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845632925 Thank you, @itholic and @HyukjinKwon . The refactoring idea looks good to me. I commented only a technical issue about the link usage. I'll leave this to @HyukjinKw

[GitHub] [spark] dongjoon-hyun commented on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845632925 Thank you, @itholic and @HyukjinKwon . The refactoring idea looks good to me. I commented only a technical issue about the link usage. -- This is an automated message fr

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636615565 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -874,23 +874,10 @@ class DataFrameReader private[sql](sparkSe

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636614962 ## File path: docs/sql-data-sources-orc.md ## @@ -172,3 +172,29 @@ When reading from Hive metastore ORC tables and inserting to Hive metastore ORC

[GitHub] [spark] SparkQA commented on pull request #32586: [SPARK-35439][SQL] Children subexpr should come first than parent subexpr

SparkQA commented on pull request #32586: URL: https://github.com/apache/spark/pull/32586#issuecomment-845632152 **[Test build #138770 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138770/testReport)** for PR 32586 at commit [`3819bf3`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636615149 ## File path: python/pyspark/sql/readwriter.py ## @@ -793,28 +793,13 @@ def orc(self, path, mergeSchema=None, pathGlobFilter=None, recursiveFileLoo

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636614962 ## File path: docs/sql-data-sources-orc.md ## @@ -172,3 +172,29 @@ When reading from Hive metastore ORC tables and inserting to Hive metastore ORC

[GitHub] [spark] yaooqinn commented on a change in pull request #32610: [SPARK-35460][K8S] invalid `spark.kubernetes.executor.podNamePrefix` causes app to hang

yaooqinn commented on a change in pull request #32610: URL: https://github.com/apache/spark/pull/32610#discussion_r636615055 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -250,11 +250,21 @@ private[spark] object C

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on a change in pull request #32546: URL: https://github.com/apache/spark/pull/32546#discussion_r636614962 ## File path: docs/sql-data-sources-orc.md ## @@ -172,3 +172,29 @@ When reading from Hive metastore ORC tables and inserting to Hive metastore ORC

[GitHub] [spark] dongjoon-hyun commented on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

dongjoon-hyun commented on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845630052 Thank you for pinging me, @HyukjinKwon . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] SparkQA commented on pull request #32546: [SPARK-35395][DOCS] Move ORC data source options from Python and Scala into a single page

SparkQA commented on pull request #32546: URL: https://github.com/apache/spark/pull/32546#issuecomment-845629618 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43300/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32589: [SPARK-35444][SQL] Imporve the logic of createTable if table already exist and ignoreIfExists=true

HyukjinKwon commented on a change in pull request #32589: URL: https://github.com/apache/spark/pull/32589#discussion_r636613685 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala ## @@ -367,6 +367,7 @@ class SessionCatalog(

  1   2   3   4   5   6   7   8   >