[GitHub] [spark] cloud-fan commented on a change in pull request #33284: [SPARK-36063][SQL] Optimize OneRowRelation subqueries

cloud-fan commented on a change in pull request #33284: URL: https://github.com/apache/spark/pull/33284#discussion_r670986145 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/QueryPlan.scala ## @@ -435,6 +435,28 @@ abstract class

[GitHub] [spark] SparkQA removed a comment on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

SparkQA removed a comment on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881133738 **[Test build #141113 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141113/testReport)** for PR 33383 at commit

[GitHub] [spark] SparkQA commented on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

SparkQA commented on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881197593 **[Test build #141113 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141113/testReport)** for PR 33383 at commit

[GitHub] [spark] SparkQA commented on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

SparkQA commented on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881196469 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45640/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-881193075 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45636/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

SparkQA removed a comment on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881133831 **[Test build #141120 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141120/testReport)** for PR 33358 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

AmplabJenkins removed a comment on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881192324 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141120/

[GitHub] [spark] AmplabJenkins commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

AmplabJenkins commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881192324 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141120/ -- This

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-881191734 **[Test build #141137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141137/testReport)** for PR 31517 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

cloud-fan commented on a change in pull request #33341: URL: https://github.com/apache/spark/pull/33341#discussion_r670979148 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala ## @@ -17,184 +17,249 @@ package org.apache.spark.sql

[GitHub] [spark] SparkQA commented on pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

SparkQA commented on pull request #33286: URL: https://github.com/apache/spark/pull/33286#issuecomment-881191179 **[Test build #141136 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141136/testReport)** for PR 33286 at commit

[GitHub] [spark] SparkQA commented on pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

SparkQA commented on pull request #33324: URL: https://github.com/apache/spark/pull/33324#issuecomment-881191177 **[Test build #141135 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141135/testReport)** for PR 33324 at commit

[GitHub] [spark] SparkQA commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

SparkQA commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881191229 **[Test build #141120 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141120/testReport)** for PR 33358 at commit

[GitHub] [spark] SparkQA commented on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

SparkQA commented on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881191119 **[Test build #141134 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141134/testReport)** for PR 33379 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

AmplabJenkins removed a comment on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881190249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45633/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33384: [SPARK-36167][PYTHON][3.2] Revisit more InternalField managements

AmplabJenkins removed a comment on pull request #33384: URL: https://github.com/apache/spark/pull/33384#issuecomment-881190252 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45639/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33363: [SPARK-36156][SQL] SCRIPT TRANSFORM ROW FORMAT DELIMITED should respect `NULL DEFINED AS` and default value should be `\N`

AmplabJenkins removed a comment on pull request #33363: URL: https://github.com/apache/spark/pull/33363#issuecomment-881190248 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45632/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

AmplabJenkins removed a comment on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881190247 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45631/

[GitHub] [spark] AmplabJenkins commented on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

AmplabJenkins commented on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881190247 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45631/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33384: [SPARK-36167][PYTHON][3.2] Revisit more InternalField managements

AmplabJenkins commented on pull request #33384: URL: https://github.com/apache/spark/pull/33384#issuecomment-881190252 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45639/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33363: [SPARK-36156][SQL] SCRIPT TRANSFORM ROW FORMAT DELIMITED should respect `NULL DEFINED AS` and default value should be `\N`

AmplabJenkins commented on pull request #33363: URL: https://github.com/apache/spark/pull/33363#issuecomment-881190248 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45632/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

AmplabJenkins commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881190249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45633/ --

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

HyukjinKwon commented on a change in pull request #33379: URL: https://github.com/apache/spark/pull/33379#discussion_r670977741 ## File path: python/pyspark/pandas/namespace.py ## @@ -2822,6 +2823,9 @@ def broadcast(obj: DataFrame) -> DataFrame: """ Marks a DataFrame

[GitHub] [spark] SparkQA commented on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

SparkQA commented on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881189875 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45637/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33384: [SPARK-36167][PYTHON][3.2] Revisit more InternalField managements

SparkQA commented on pull request #33384: URL: https://github.com/apache/spark/pull/33384#issuecomment-881189460 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45639/ --

[GitHub] [spark] dongjoon-hyun commented on pull request #33356: [SPARK-36146][PYTHON][INFRA][TESTS] Upgrade Python version from 3.6 to 3.9 in GitHub Actions' linter/docs

dongjoon-hyun commented on pull request #33356: URL: https://github.com/apache/spark/pull/33356#issuecomment-881189270 No problem~ Actually, it was my bad which didn't check it clearly at this PR. ;) -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33356: [SPARK-36146][PYTHON][INFRA][TESTS] Upgrade Python version from 3.6 to 3.9 in GitHub Actions' linter/docs

dongjoon-hyun edited a comment on pull request #33356: URL: https://github.com/apache/spark/pull/33356#issuecomment-881184653 Ur, @HyukjinKwon . This requires `[SPARK-36165][INFRA] Fix SQL doc generation in GitHub Action`. I'll backport it, too. -- This is an automated message from

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33378: [SPARK-32922][SHUFFLE][CORE] Fixes few issues when the executor tries to fetch push-merged blocks

dongjoon-hyun commented on a change in pull request #33378: URL: https://github.com/apache/spark/pull/33378#discussion_r670975694 ## File path: core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala ## @@ -197,9 +197,12 @@ private class PushBasedFetchHelper(

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33378: [SPARK-32922][SHUFFLE][CORE] Fixes few issues when the executor tries to fetch push-merged blocks

dongjoon-hyun commented on a change in pull request #33378: URL: https://github.com/apache/spark/pull/33378#discussion_r670975574 ## File path: core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala ## @@ -197,9 +197,12 @@ private class PushBasedFetchHelper(

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33378: [SPARK-32922][SHUFFLE][CORE] Fixes few issues when the executor tries to fetch push-merged blocks

dongjoon-hyun commented on a change in pull request #33378: URL: https://github.com/apache/spark/pull/33378#discussion_r670975288 ## File path: core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala ## @@ -197,9 +197,12 @@ private class PushBasedFetchHelper(

[GitHub] [spark] SparkQA commented on pull request #33227: [SPARK-35972][SQL][3.1] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

SparkQA commented on pull request #33227: URL: https://github.com/apache/spark/pull/33227#issuecomment-881186988 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45635/ -- This is an automated message from the Apache

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

AngersZh commented on a change in pull request #33324: URL: https://github.com/apache/spark/pull/33324#discussion_r670974519 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -4058,6 +4058,44 @@ class SQLQuerySuite extends QueryTest with

[GitHub] [spark] HyukjinKwon commented on pull request #33356: [SPARK-36146][PYTHON][INFRA][TESTS] Upgrade Python version from 3.6 to 3.9 in GitHub Actions' linter/docs

HyukjinKwon commented on pull request #33356: URL: https://github.com/apache/spark/pull/33356#issuecomment-881186206 oops. thanks you @dongjoon-hyun. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] LuciferYang closed pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

LuciferYang closed pull request #33267: URL: https://github.com/apache/spark/pull/33267 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] LuciferYang commented on a change in pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

LuciferYang commented on a change in pull request #33267: URL: https://github.com/apache/spark/pull/33267#discussion_r670973736 ## File path: core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala ## @@ -288,3 +288,19 @@ private[spark] class

[GitHub] [spark] dongjoon-hyun commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881185404 The UT failure is irrelevant to this PR because this PR is only adding a new GitHub Action. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] dongjoon-hyun commented on pull request #33356: [SPARK-36146][PYTHON][INFRA][TESTS] Upgrade Python version from 3.6 to 3.9 in GitHub Actions' linter/docs

dongjoon-hyun commented on pull request #33356: URL: https://github.com/apache/spark/pull/33356#issuecomment-881184653 Ur, @HyukjinKwon . This requires `[SPARK-36165][INFRA] Fix SQL doc generation in GitHub Action`. I'll backport it. -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

SparkQA commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881184090 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45633/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun commented on a change in pull request #33358: URL: https://github.com/apache/spark/pull/33358#discussion_r670971606 ## File path: .github/workflows/build_and_test_scala213_daily.yml ## @@ -0,0 +1,115 @@ +name: Build and test Scala 2.13 daily + +on: + schedule: +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun commented on a change in pull request #33358: URL: https://github.com/apache/spark/pull/33358#discussion_r670971381 ## File path: .github/workflows/build_and_test_scala213_daily.yml ## @@ -0,0 +1,115 @@ +name: Build and test Scala 2.13 daily + +on: + schedule: +

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun commented on a change in pull request #33358: URL: https://github.com/apache/spark/pull/33358#discussion_r670971006 ## File path: .github/workflows/build_and_test_scala213_daily.yml ## @@ -0,0 +1,115 @@ +name: Build and test Scala 2.13 daily + +on: + schedule: +

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun edited a comment on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881182013 Hi, @HyukjinKwon , @sarutak , @gengliangwang . This PR is ready and tested in my repo. I updated the PR description. This PR is only about adding a daily

[GitHub] [spark] dongjoon-hyun commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

dongjoon-hyun commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881182013 Hi, @HyukjinKwon , @sarutak , @gengliangwang . This PR is ready and tested in my repo. I updated the PR description. -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33363: [SPARK-36156][SQL] SCRIPT TRANSFORM ROW FORMAT DELIMITED should respect `NULL DEFINED AS` and default value should be `\N`

SparkQA commented on pull request #33363: URL: https://github.com/apache/spark/pull/33363#issuecomment-881179134 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45632/ -- This is an automated message from the

[GitHub] [spark] zhouyejoe commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

zhouyejoe commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r670966938 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -821,20 +914,132 @@ public

[GitHub] [spark] zhouyejoe commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

zhouyejoe commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r670966164 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -778,11 +773,6 @@ public void

[GitHub] [spark] SparkQA commented on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

SparkQA commented on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881177390 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45631/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

cloud-fan commented on a change in pull request #33286: URL: https://github.com/apache/spark/pull/33286#discussion_r670966052 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala ## @@ -119,6 +120,18 @@ case class ColumnStat(

[GitHub] [spark] karenfeng commented on a change in pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

karenfeng commented on a change in pull request #33286: URL: https://github.com/apache/spark/pull/33286#discussion_r670965165 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala ## @@ -119,6 +120,18 @@ case class ColumnStat(

[GitHub] [spark] AmplabJenkins commented on pull request #33284: [SPARK-36063][SQL] Optimize OneRowRelation subqueries

AmplabJenkins commented on pull request #33284: URL: https://github.com/apache/spark/pull/33284#issuecomment-881175656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141108/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33284: [SPARK-36063][SQL] Optimize OneRowRelation subqueries

AmplabJenkins removed a comment on pull request #33284: URL: https://github.com/apache/spark/pull/33284#issuecomment-881175656 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141108/

[GitHub] [spark] cloud-fan edited a comment on pull request #33191: [SPARK-35985][SQL] push partitionFilters for empty readDataSchema

cloud-fan edited a comment on pull request #33191: URL: https://github.com/apache/spark/pull/33191#issuecomment-881175486 thanks, merging to master/3.2 (it's more like a bug fix for ds v2 pushdown)! -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

AmplabJenkins removed a comment on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881175473 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45638/

[GitHub] [spark] cloud-fan closed pull request #33191: [SPARK-35985][SQL] push partitionFilters for empty readDataSchema

cloud-fan closed pull request #33191: URL: https://github.com/apache/spark/pull/33191 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33191: [SPARK-35985][SQL] push partitionFilters for empty readDataSchema

cloud-fan commented on pull request #33191: URL: https://github.com/apache/spark/pull/33191#issuecomment-881175486 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AmplabJenkins commented on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

AmplabJenkins commented on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881175473 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45638/ --

[GitHub] [spark] SparkQA commented on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

SparkQA commented on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881175463 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45638/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33284: [SPARK-36063][SQL] Optimize OneRowRelation subqueries

SparkQA removed a comment on pull request #33284: URL: https://github.com/apache/spark/pull/33284#issuecomment-881079782 **[Test build #141108 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141108/testReport)** for PR 33284 at commit

[GitHub] [spark] SparkQA commented on pull request #33284: [SPARK-36063][SQL] Optimize OneRowRelation subqueries

SparkQA commented on pull request #33284: URL: https://github.com/apache/spark/pull/33284#issuecomment-881174823 **[Test build #141108 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141108/testReport)** for PR 33284 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

cloud-fan commented on a change in pull request #33286: URL: https://github.com/apache/spark/pull/33286#discussion_r670962182 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/EstimationUtils.scala ## @@ -52,14 +52,18 @@

[GitHub] [spark] cloud-fan commented on a change in pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

cloud-fan commented on a change in pull request #33286: URL: https://github.com/apache/spark/pull/33286#discussion_r670962058 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala ## @@ -119,6 +120,18 @@ case class ColumnStat(

[GitHub] [spark] cloud-fan commented on a change in pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

cloud-fan commented on a change in pull request #33286: URL: https://github.com/apache/spark/pull/33286#discussion_r670961773 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/Statistics.scala ## @@ -119,6 +120,18 @@ case class ColumnStat(

[GitHub] [spark] SparkQA commented on pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

SparkQA commented on pull request #33286: URL: https://github.com/apache/spark/pull/33286#issuecomment-881172394 **[Test build #141133 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141133/testReport)** for PR 33286 at commit

[GitHub] [spark] SparkQA commented on pull request #33358: [SPARK-36152][INFRA][TESTS] Add Scala 2.13 daily build and test GitHub Action job

SparkQA commented on pull request #33358: URL: https://github.com/apache/spark/pull/33358#issuecomment-881172275 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45633/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

SparkQA commented on pull request #33341: URL: https://github.com/apache/spark/pull/33341#issuecomment-881170478 **[Test build #141132 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141132/testReport)** for PR 33341 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

AmplabJenkins removed a comment on pull request #33324: URL: https://github.com/apache/spark/pull/33324#issuecomment-881170182 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45634/

[GitHub] [spark] AmplabJenkins commented on pull request #33385: [WIP][SPARK-36173][CORE] Support getting CPU number in TaskContext

AmplabJenkins commented on pull request #33385: URL: https://github.com/apache/spark/pull/33385#issuecomment-881170304 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

AmplabJenkins removed a comment on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881170080 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45626/

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33291: [SPARK-35561][SQL] Remove leading zeros from empty static number type partition

HyukjinKwon commented on a change in pull request #33291: URL: https://github.com/apache/spark/pull/33291#discussion_r670959133 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala ## @@ -351,10 +351,20 @@ object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33382: [SPARK-36137][SQL] HiveShim should fallback to getAllPartitionsOf even if directSQL is enabled in remote HMS

AmplabJenkins removed a comment on pull request #33382: URL: https://github.com/apache/spark/pull/33382#issuecomment-881170081 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45627/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33381: [SPARK-36169][SQL] Make 'spark.sql.sources.disabledJdbcConnProviderList' as a static conf (as documneted)

AmplabJenkins removed a comment on pull request #33381: URL: https://github.com/apache/spark/pull/33381#issuecomment-881170078 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45628/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33380: [SPARK-36170][SQL] Change quoted interval literal (interval constructor) to be converted to ANSI interval types

AmplabJenkins removed a comment on pull request #33380: URL: https://github.com/apache/spark/pull/33380#issuecomment-881170079 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45629/

[GitHub] [spark] AmplabJenkins commented on pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

AmplabJenkins commented on pull request #33324: URL: https://github.com/apache/spark/pull/33324#issuecomment-881170182 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45634/ --

[GitHub] [spark] SparkQA commented on pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

SparkQA commented on pull request #33324: URL: https://github.com/apache/spark/pull/33324#issuecomment-881170170 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45634/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33382: [SPARK-36137][SQL] HiveShim should fallback to getAllPartitionsOf even if directSQL is enabled in remote HMS

AmplabJenkins commented on pull request #33382: URL: https://github.com/apache/spark/pull/33382#issuecomment-881170081 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45627/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33381: [SPARK-36169][SQL] Make 'spark.sql.sources.disabledJdbcConnProviderList' as a static conf (as documneted)

AmplabJenkins commented on pull request #33381: URL: https://github.com/apache/spark/pull/33381#issuecomment-881170078 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45628/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33380: [SPARK-36170][SQL] Change quoted interval literal (interval constructor) to be converted to ANSI interval types

AmplabJenkins commented on pull request #33380: URL: https://github.com/apache/spark/pull/33380#issuecomment-881170079 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45629/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

AmplabJenkins commented on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881170080 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45626/ --

[GitHub] [spark] dgd-contributor commented on pull request #33291: [SPARK-35561][SQL] Remove leading zeros from empty static number type partition

dgd-contributor commented on pull request #33291: URL: https://github.com/apache/spark/pull/33291#issuecomment-881168482 @srowen @HyukjinKwon Do you think this change is ready to merge? -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] karenfeng commented on pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

karenfeng commented on pull request #33286: URL: https://github.com/apache/spark/pull/33286#issuecomment-881167512 Taking a look now, thanks for checking! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #33363: [SPARK-36156][SQL] SCRIPT TRANSFORM ROW FORMAT DELIMITED should respect `NULL DEFINED AS` and default value should be `\N`

SparkQA commented on pull request #33363: URL: https://github.com/apache/spark/pull/33363#issuecomment-881167552 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45632/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

cloud-fan commented on pull request #33286: URL: https://github.com/apache/spark/pull/33286#issuecomment-881166830 `TPCDSModifiedPlanStabilityWithStatsSuite` fails... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

SparkQA commented on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881166143 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45626/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33380: [SPARK-36170][SQL] Change quoted interval literal (interval constructor) to be converted to ANSI interval types

SparkQA commented on pull request #33380: URL: https://github.com/apache/spark/pull/33380#issuecomment-881165684 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45629/ -- This is an automated message from the

[GitHub] [spark] beliefer commented on a change in pull request #33341: [SPARK-36091][SQL] Support TimestampNTZ type in expression TimeWindow

beliefer commented on a change in pull request #33341: URL: https://github.com/apache/spark/pull/33341#discussion_r670953749 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameTimeWindowingSuite.scala ## @@ -27,177 +29,234 @@ class DataFrameTimeWindowingSuite

[GitHub] [spark] xwu99 opened a new pull request #33385: [WIP][SPARK-36173][CORE] Support getting CPU number in TaskContext

xwu99 opened a new pull request #33385: URL: https://github.com/apache/spark/pull/33385 In stage-level resource scheduling, the allocated 3rd party resources can be obtained in TaskContext using resources() interface, however there is no API to get how many cpus are allocated for the

[GitHub] [spark] SparkQA commented on pull request #33373: [SPARK-36127][PYTHON] Support comparison between a Categorical and a scalar

SparkQA commented on pull request #33373: URL: https://github.com/apache/spark/pull/33373#issuecomment-881163476 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45631/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33382: [SPARK-36137][SQL] HiveShim should fallback to getAllPartitionsOf even if directSQL is enabled in remote HMS

SparkQA commented on pull request #33382: URL: https://github.com/apache/spark/pull/33382#issuecomment-881162571 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45627/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33381: [SPARK-36169][SQL] Make 'spark.sql.sources.disabledJdbcConnProviderList' as a static conf (as documneted)

SparkQA commented on pull request #33381: URL: https://github.com/apache/spark/pull/33381#issuecomment-881162168 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45628/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on pull request #32861: [SPARK-35710] [SQL] Support DPP + AQE when there is no reused broadcast exchange

cloud-fan commented on pull request #32861: URL: https://github.com/apache/spark/pull/32861#issuecomment-881161192 @JkSelf can you check the test failures? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on a change in pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

cloud-fan commented on a change in pull request #33324: URL: https://github.com/apache/spark/pull/33324#discussion_r670949390 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -4058,6 +4058,44 @@ class SQLQuerySuite extends QueryTest with

[GitHub] [spark] HyukjinKwon commented on pull request #33384: [SPARK-36167][PYTHON][3.2] Revisit more InternalField managements

HyukjinKwon commented on pull request #33384: URL: https://github.com/apache/spark/pull/33384#issuecomment-881153833 looks like there's a valid test failure: ``` == ERROR [1.825s]: test_intersection

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33081: [SPARK-34893][SS] Support session window natively

AmplabJenkins removed a comment on pull request #33081: URL: https://github.com/apache/spark/pull/33081#issuecomment-881138887 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45623/

[GitHub] [spark] SparkQA removed a comment on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

SparkQA removed a comment on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881152936 **[Test build #141129 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141129/testReport)** for PR 33379 at commit

[GitHub] [spark] SparkQA commented on pull request #33380: [SPARK-36170][SQL] Change quoted interval literal (interval constructor) to be converted to ANSI interval types

SparkQA commented on pull request #33380: URL: https://github.com/apache/spark/pull/33380#issuecomment-881153740 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45629/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

AmplabJenkins removed a comment on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881153316 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141129/

[GitHub] [spark] SparkQA commented on pull request #33383: [SPARK-36171][BUILD] Upgrade GenJavadoc to 0.18

SparkQA commented on pull request #33383: URL: https://github.com/apache/spark/pull/33383#issuecomment-881153606 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45626/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

AmplabJenkins commented on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881153316 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141129/ -- This

[GitHub] [spark] SparkQA commented on pull request #33379: [SPARK-35810][PYTHON] Deprecate ps.broadcast API

SparkQA commented on pull request #33379: URL: https://github.com/apache/spark/pull/33379#issuecomment-881153305 **[Test build #141129 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141129/testReport)** for PR 33379 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #33324: [SPARK-36093][SQL] RemoveRedundantAliases should not change Command's parameter's expression's name

AngersZh commented on pull request #33324: URL: https://github.com/apache/spark/pull/33324#issuecomment-881153226 > @AngersZh, it would be great to describe the user-facing behviour change. Copy the reproducer in the JIRA, and show the results before/after this PR. Updated

  1   2   3   4   5   6   7   8   9   10   >