[GitHub] [spark] SparkQA removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837717297 **[Test build #138353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138353/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837873050 **[Test build #138353 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138353/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

SparkQA removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837593393 **[Test build #138348 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138348/testReport)** for PR 32303 at commit

[GitHub] [spark] SparkQA commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

SparkQA commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837870821 **[Test build #138348 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138348/testReport)** for PR 32303 at commit

[GitHub] [spark] beliefer commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

beliefer commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837864187 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

SparkQA removed a comment on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837716519 **[Test build #138352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138352/testReport)** for PR 32464 at commit

[GitHub] [spark] SparkQA commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

SparkQA commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837856847 **[Test build #138352 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138352/testReport)** for PR 32464 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA removed a comment on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837593024 **[Test build #138347 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138347/testReport)** for PR 32482 at commit

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837853557 **[Test build #138347 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138347/testReport)** for PR 32482 at commit

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837836414 **[Test build #138359 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138359/testReport)** for PR 32399 at commit

[GitHub] [spark] SparkQA commented on pull request #32470: [WIP] Simplify ResolveAggregateFunctions

SparkQA commented on pull request #32470: URL: https://github.com/apache/spark/pull/32470#issuecomment-837836239 **[Test build #138358 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138358/testReport)** for PR 32470 at commit

[GitHub] [spark] SparkQA commented on pull request #32497: [SPARK-35366][SQL] Avoid using deprecated `buildForBatch` and `buildForStreaming`

SparkQA commented on pull request #32497: URL: https://github.com/apache/spark/pull/32497#issuecomment-837836024 **[Test build #138357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138357/testReport)** for PR 32497 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

AmplabJenkins removed a comment on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-837833240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138346/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins removed a comment on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837833239 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42877/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

AmplabJenkins removed a comment on pull request #31986: URL: https://github.com/apache/spark/pull/31986#issuecomment-837833241 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42878/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still co

AmplabJenkins removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837833238 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138350/

[GitHub] [spark] AmplabJenkins commented on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

AmplabJenkins commented on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-837833240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138346/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue r

AmplabJenkins commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837833238 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138350/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

AmplabJenkins commented on pull request #31986: URL: https://github.com/apache/spark/pull/31986#issuecomment-837833241 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42878/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837833239 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42877/ --

[GitHub] [spark] gerashegalov commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

gerashegalov commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-837830422 @zhengruifeng can you provide a minimum code reproducing for NPEs you are observing? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] venkata91 commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629848900 ## File path: .idea/vcs.xml ## @@ -1,36 +0,0 @@ - Review comment: Sorry my bad, I think it got added as part of this `[SPARK-35223] Add

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837823698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

SparkQA commented on pull request #31986: URL: https://github.com/apache/spark/pull/31986#issuecomment-837818648 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] venkata91 commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629845232 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1271,21 +1302,28 @@ private[spark] class DAGScheduler( *

[GitHub] [spark] venkata91 commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629844923 ## File path: .idea/vcs.xml ## @@ -1,36 +0,0 @@ - Review comment: Somehow my idea file got added and pushed. I think I removed it. Isn't it?

[GitHub] [spark] SparkQA removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue

SparkQA removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837652969 **[Test build #138350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138350/testReport)** for PR 32399 at commit

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837803879 **[Test build #138350 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138350/testReport)** for PR 32399 at commit

[GitHub] [spark] linhongliu-db opened a new pull request #32497: [SPARK-35366][SQL] Avoid using deprecated `buildForBatch` and `buildForStreaming`

linhongliu-db opened a new pull request #32497: URL: https://github.com/apache/spark/pull/32497 ### What changes were proposed in this pull request? Currently, in DSv2, we are still using the deprecated `buildForBatch` and `buildForStreaming`. This PR implements the `build`,

[GitHub] [spark] SparkQA removed a comment on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

SparkQA removed a comment on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-837543445 **[Test build #138346 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138346/testReport)** for PR 32457 at commit

[GitHub] [spark] SparkQA commented on pull request #32457: [SPARK-35329][SQL] Split generated switch code into pieces in ExpandExec

SparkQA commented on pull request #32457: URL: https://github.com/apache/spark/pull/32457#issuecomment-837793182 **[Test build #138346 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138346/testReport)** for PR 32457 at commit

[GitHub] [spark] SparkQA commented on pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

SparkQA commented on pull request #31986: URL: https://github.com/apache/spark/pull/31986#issuecomment-837775575 **[Test build #138356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138356/testReport)** for PR 31986 at commit

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837774793 **[Test build #138355 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138355/testReport)** for PR 32482 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToPrettyString for keep consistent between hive/spark format in df.show and transform

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-837771490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42876/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

AmplabJenkins removed a comment on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837771494 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138343/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

AmplabJenkins removed a comment on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837771487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42875/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837771488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42874/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

AmplabJenkins removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837771486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42873/

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToPrettyString for keep consistent between hive/spark format in df.show and transform

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-837771490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42876/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

AmplabJenkins commented on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837771494 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138343/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

AmplabJenkins commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837771487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42875/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837771488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42874/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

AmplabJenkins commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837771486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42873/ --

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToPrettyString for keep consistent between hive/spark format in df.show and transform

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-837768084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

SparkQA commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837765641 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42875/ -- This is an automated message from the

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

HeartSaVioR commented on a change in pull request #31986: URL: https://github.com/apache/spark/pull/31986#discussion_r629830829 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/UpdatingSessionsExec.scala ## @@ -0,0 +1,77 @@ +/* + * Licensed to

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

HeartSaVioR commented on a change in pull request #31986: URL: https://github.com/apache/spark/pull/31986#discussion_r629830829 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/UpdatingSessionsExec.scala ## @@ -0,0 +1,77 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

SparkQA commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837759960 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42875/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837756097 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837755790 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42874/ --

[GitHub] [spark] sigmod commented on a change in pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

sigmod commented on a change in pull request #32439: URL: https://github.com/apache/spark/pull/32439#discussion_r629828096 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -745,7 +754,8 @@ object

[GitHub] [spark] maryannxue commented on a change in pull request #32439: [SPARK-35298][SQL] Migrate to transformWithPruning for rules in Optimizer.scala

maryannxue commented on a change in pull request #32439: URL: https://github.com/apache/spark/pull/32439#discussion_r629826756 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -745,7 +754,8 @@ object

[GitHub] [spark] SparkQA removed a comment on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

SparkQA removed a comment on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837480412 **[Test build #138343 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138343/testReport)** for PR 32495 at commit

[GitHub] [spark] SparkQA commented on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

SparkQA commented on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837741723 **[Test build #138343 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138343/testReport)** for PR 32495 at commit

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31986: [SPARK-34888][SS] Introduce UpdatingSessionIterator adjusting session window on elements

HeartSaVioR commented on a change in pull request #31986: URL: https://github.com/apache/spark/pull/31986#discussion_r629795680 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/UpdatingSessionsExec.scala ## @@ -0,0 +1,77 @@ +/* + * Licensed to

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToPrettyString for keep consistent between hive/spark format in df.show and transform

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-837727960 **[Test build #138354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138354/testReport)** for PR 32365 at commit

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-837717297 **[Test build #138353 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138353/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837716583 **[Test build #138351 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138351/testReport)** for PR 32494 at commit

[GitHub] [spark] SparkQA commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

SparkQA commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837716519 **[Test build #138352 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138352/testReport)** for PR 32464 at commit

[GitHub] [spark] beliefer commented on a change in pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

beliefer commented on a change in pull request #32464: URL: https://github.com/apache/spark/pull/32464#discussion_r629818005 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryCompilationErrors.scala ## @@ -1391,4 +1391,58 @@ private[spark] object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still co

AmplabJenkins removed a comment on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837712777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42872/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins removed a comment on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837670869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue r

AmplabJenkins commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837712777 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42872/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837712776 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138349/ -- This

[GitHub] [spark] beliefer commented on pull request #32464: [SPARK-35062][SQL] Group exception messages in sql/streaming

beliefer commented on pull request #32464: URL: https://github.com/apache/spark/pull/32464#issuecomment-837711621 > Looks good! There are two more exceptions under `streaming/ui`. How about adding them in the same PR? > > 'sql/core/src/main/scala/org/apache/spark/sql/streaming/ui'

[GitHub] [spark] SparkQA removed a comment on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA removed a comment on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837604703 **[Test build #138349 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138349/testReport)** for PR 32482 at commit

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837709031 **[Test build #138349 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138349/testReport)** for PR 32482 at commit

[GitHub] [spark] cfmcgrady commented on a change in pull request #32488: [SPARK-35316][SQL] UnwrapCastInBinaryComparison support In/InSet predicate

cfmcgrady commented on a change in pull request #32488: URL: https://github.com/apache/spark/pull/32488#discussion_r629816464 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/UnwrapCastInBinaryComparison.scala ## @@ -89,10 +89,11 @@ import

[GitHub] [spark] zhengruifeng commented on pull request #31540: [SPARK-20977][CORE] Use a non-final field for the state of CollectionAccumulator

zhengruifeng commented on pull request #31540: URL: https://github.com/apache/spark/pull/31540#issuecomment-837702523 > This does not necessarily solve the issue that @zsxwing detailed - the issue here is `registerAccumulator` should not be called in `readObject` before subclasses have

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837699020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629813442 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2004,6 +2020,131 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629812883 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -1271,21 +1302,28 @@ private[spark] class DAGScheduler( *

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629811689 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2004,6 +2020,131 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] LuciferYang commented on a change in pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

LuciferYang commented on a change in pull request #32455: URL: https://github.com/apache/spark/pull/32455#discussion_r629811420 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala ## @@ -1434,9 +1435,10 @@ object

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629810826 ## File path: core/src/main/scala/org/apache/spark/Dependency.scala ## @@ -110,6 +125,12 @@ class ShuffleDependency[K: ClassTag, V: ClassTag, C:

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629810563 ## File path: core/src/main/scala/org/apache/spark/Dependency.scala ## @@ -96,12 +96,27 @@ class ShuffleDependency[K: ClassTag, V: ClassTag, C:

[GitHub] [spark] mridulm commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

mridulm commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r629810296 ## File path: .idea/vcs.xml ## @@ -1,36 +0,0 @@ - Review comment: Where is this coming from ? -- This is an automated message from the

[GitHub] [spark] mridulm commented on pull request #32381: [SPARK-35229][WEBUI] Limit the maximum number of items on the timeline view.

mridulm commented on pull request #32381: URL: https://github.com/apache/spark/pull/32381#issuecomment-837683190 +CC @zhouyejoe -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] shahidki31 commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

shahidki31 commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837680984 Retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AmplabJenkins commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837670869 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42871/ --

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837670817 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42871/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32204: [SPARK-34494][SQL][DOCS] Move JSON data source options from Python and Scala into a single page

HyukjinKwon commented on a change in pull request #32204: URL: https://github.com/apache/spark/pull/32204#discussion_r629803248 ## File path: python/pyspark/sql/streaming.py ## @@ -504,105 +504,13 @@ def json(self, path, schema=None, primitivesAsString=None,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32204: [SPARK-34494][SQL][DOCS] Move JSON data source options from Python and Scala into a single page

HyukjinKwon commented on a change in pull request #32204: URL: https://github.com/apache/spark/pull/32204#discussion_r629803050 ## File path: python/pyspark/sql/readwriter.py ## @@ -233,114 +233,13 @@ def json(self, path, schema=None, primitivesAsString=None,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

HyukjinKwon commented on a change in pull request #32161: URL: https://github.com/apache/spark/pull/32161#discussion_r629802818 ## File path: python/pyspark/sql/readwriter.py ## @@ -416,53 +416,10 @@ def parquet(self, *paths, **options): Other Parameters

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32161: [SPARK-35025][SQL][PYTHON][DOCS] Move Parquet data source options from Python and Scala into a single page.

HyukjinKwon commented on a change in pull request #32161: URL: https://github.com/apache/spark/pull/32161#discussion_r629802560 ## File path: python/pyspark/sql/readwriter.py ## @@ -1257,14 +1214,13 @@ def parquet(self, path, mode=None, partitionBy=None, compression=None):

[GitHub] [spark] c21 commented on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

c21 commented on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837659022 Thank you @maropu for monitoring and review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] maropu commented on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

maropu commented on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837658640 All the GA tests passed. Merged to master. Thank you, @c21 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] maropu closed pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

maropu closed pull request #32495: URL: https://github.com/apache/spark/pull/32495 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] SparkQA commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

SparkQA commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837656482 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42871/ -- This is an automated message from the Apache

[GitHub] [spark] maropu commented on pull request #32495: [SPARK-35363][SQL] Refactor sort merge join code-gen be agnostic to join type

maropu commented on pull request #32495: URL: https://github.com/apache/spark/pull/32495#issuecomment-837656333 okay, I've checked that it passed. I will merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

AmplabJenkins removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837655724 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138342/

[GitHub] [spark] AmplabJenkins commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

AmplabJenkins commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837655724 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/138342/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

SparkQA removed a comment on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837428314 **[Test build #138342 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138342/testReport)** for PR 32494 at commit

[GitHub] [spark] SparkQA commented on pull request #32494: [Minor][SPARK-35362][SQL]Update null count in the column stats for UNION operator stats estimation

SparkQA commented on pull request #32494: URL: https://github.com/apache/spark/pull/32494#issuecomment-837654038 **[Test build #138342 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138342/testReport)** for PR 32494 at commit

[GitHub] [spark] SparkQA commented on pull request #32399: [SPARK-35271][ML][PYSPARK] Fix: After CrossValidator/TrainValidationSplit fit raised error, some backgroud threads may still continue run or

SparkQA commented on pull request #32399: URL: https://github.com/apache/spark/pull/32399#issuecomment-837652969 **[Test build #138350 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/138350/testReport)** for PR 32399 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

AmplabJenkins removed a comment on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837651747 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42870/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins removed a comment on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837651748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42869/

[GitHub] [spark] AmplabJenkins commented on pull request #32482: [SPARK-35332][SQL] Make cache plan disable configs configurable

AmplabJenkins commented on pull request #32482: URL: https://github.com/apache/spark/pull/32482#issuecomment-837651748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42869/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32303: [SPARK-34382][SQL] Support LATERAL subqueries

AmplabJenkins commented on pull request #32303: URL: https://github.com/apache/spark/pull/32303#issuecomment-837651747 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42870/ --

[GitHub] [spark] beliefer commented on pull request #32492: [SPARK-35088][SQL][FOLLOWUP] Improve the error message for Sequence expression

beliefer commented on pull request #32492: URL: https://github.com/apache/spark/pull/32492#issuecomment-837647119 @HyukjinKwon @MaxGekk Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

  1   2   3   4   5   >