[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891555119 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46493/ -- This is an automated message from the Apache

[GitHub] [spark] zhouyejoe commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

zhouyejoe commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891554593 +1. LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] zhuqi-lucas commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

zhuqi-lucas commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891552583 +1, LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins removed a comment on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891550216 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141978/

[GitHub] [spark] SparkQA removed a comment on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA removed a comment on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891501800 **[Test build #141978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141978/testReport)** for PR 33583 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891550216 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141978/ -- This

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891550056 **[Test build #141978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141978/testReport)** for PR 33583 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins removed a comment on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891549710 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46489/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32355: [SPARK-35221][SQL] Add join hint build side check

AmplabJenkins removed a comment on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-891549708 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46492/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

AmplabJenkins removed a comment on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891549706 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46488/

[GitHub] [spark] AmplabJenkins commented on pull request #32355: [SPARK-35221][SQL] Add join hint build side check

AmplabJenkins commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-891549708 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46492/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

AmplabJenkins commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891549706 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46488/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891549710 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46489/ --

[GitHub] [spark] itholic commented on a change in pull request #33581: [SPARK-36192][PYTHON] Better error messages for DataTypeOps against lists

itholic commented on a change in pull request #33581: URL: https://github.com/apache/spark/pull/33581#discussion_r681451616 ## File path: python/pyspark/pandas/data_type_ops/base.py ## @@ -314,9 +320,11 @@ def __or__(self, left: IndexOpsLike, right: Any) -> SeriesOrIndex:

[GitHub] [spark] zhuqi-lucas edited a comment on pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error m…

zhuqi-lucas edited a comment on pull request #33617: URL: https://github.com/apache/spark/pull/33617#issuecomment-891496800 cc @Ngone51 @zhouyejoe @mridulm @dongjoon-hyun @HyukjinKwon Could you help review this, thanks. -- This is an automated message from the Apache Git Service.

[GitHub] [spark] itholic commented on a change in pull request #32964: [SPARK-35811][PYTHON] Deprecate DataFrame.to_spark_io

itholic commented on a change in pull request #32964: URL: https://github.com/apache/spark/pull/32964#discussion_r681449016 ## File path: python/pyspark/pandas/frame.py ## @@ -4815,6 +4815,13 @@ def to_spark_io( index_col: Optional[Union[str, List[str]]] = None,

[GitHub] [spark] SparkQA commented on pull request #33509: [SPARK-36280][SQL] Remove redundant aliases after RewritePredicateSubquery

SparkQA commented on pull request #33509: URL: https://github.com/apache/spark/pull/33509#issuecomment-891545086 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46491/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891544065 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46490/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32355: [SPARK-35221][SQL] Add join hint build side check

SparkQA commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-891543486 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46492/ --

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891537889 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46489/ -- This is an automated message from the

[GitHub] [spark] imback82 commented on a change in pull request #33618: [SPARK-36381][SQL] Add case sensitive and case insensitive compare for checking column name exist when alter table

imback82 commented on a change in pull request #33618: URL: https://github.com/apache/spark/pull/33618#discussion_r681440489 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -407,6 +408,65 @@ trait AlterTableTests extends

[GitHub] [spark] SparkQA commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

SparkQA commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891532825 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46488/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891529512 **[Test build #141982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141982/testReport)** for PR 33583 at commit

[GitHub] [spark] HyukjinKwon closed pull request #33614: [SPARK-36367][3.2][PYTHON] Partially backport to avoid unexpected error with pandas 1.3

HyukjinKwon closed pull request #33614: URL: https://github.com/apache/spark/pull/33614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon edited a comment on pull request #33614: [SPARK-36367][3.2][PYTHON] Partially backport to avoid unexpected error with pandas 1.3

HyukjinKwon edited a comment on pull request #33614: URL: https://github.com/apache/spark/pull/33614#issuecomment-891528194 I think it's fine ... pandas on Spark will work 99.9% fine with pandas 1.3+ ... -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon commented on pull request #33614: [SPARK-36367][3.2][PYTHON] Partially backport to avoid unexpected error with pandas 1.3

HyukjinKwon commented on pull request #33614: URL: https://github.com/apache/spark/pull/33614#issuecomment-891528315 let me merge this in first anyway since RC will likely be cut out soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] HyukjinKwon commented on pull request #33614: [SPARK-36367][3.2][PYTHON] Partially backport to avoid unexpected error with pandas 1.3

HyukjinKwon commented on pull request #33614: URL: https://github.com/apache/spark/pull/33614#issuecomment-891528194 I think it's fine ... pandas on Spark will work 99% fine with pandas 1.3+ ... -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon closed pull request #33598: [SPARK-36345][SPARK-36367][INFRA][PYTHON] Disable tests failed by the incompatible behavior of pandas 1.3

HyukjinKwon closed pull request #33598: URL: https://github.com/apache/spark/pull/33598 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33598: [SPARK-36345][SPARK-36367][INFRA][PYTHON] Disable tests failed by the incompatible behavior of pandas 1.3

HyukjinKwon commented on pull request #33598: URL: https://github.com/apache/spark/pull/33598#issuecomment-891527673 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

HyukjinKwon commented on pull request #33560: URL: https://github.com/apache/spark/pull/33560#issuecomment-891526878 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon closed pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

HyukjinKwon closed pull request #33560: URL: https://github.com/apache/spark/pull/33560 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #32355: [SPARK-35221][SQL] Add join hint build side check

SparkQA commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-891523414 **[Test build #141981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141981/testReport)** for PR 32355 at commit

[GitHub] [spark] SparkQA commented on pull request #33509: [SPARK-36280][SQL] Remove redundant aliases after RewritePredicateSubquery

SparkQA commented on pull request #33509: URL: https://github.com/apache/spark/pull/33509#issuecomment-891522999 **[Test build #141980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141980/testReport)** for PR 33509 at commit

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891522923 **[Test build #141979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141979/testReport)** for PR 33583 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

AmplabJenkins removed a comment on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891522011 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46486/

[GitHub] [spark] AmplabJenkins commented on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

AmplabJenkins commented on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891522011 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46486/ --

[GitHub] [spark] SparkQA commented on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

SparkQA commented on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891521982 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46486/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33618: [SPARK-36381][SQL] Add case sensitive and case insensitive compare for checking column name exist when alter table

AmplabJenkins commented on pull request #33618: URL: https://github.com/apache/spark/pull/33618#issuecomment-891521779 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-891521259 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46483/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is receive

AmplabJenkins removed a comment on pull request #33605: URL: https://github.com/apache/spark/pull/33605#issuecomment-891521262 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46487/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

AmplabJenkins removed a comment on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891521264 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46482/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins removed a comment on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891521258 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

AmplabJenkins commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891521260 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is received

AmplabJenkins commented on pull request #33605: URL: https://github.com/apache/spark/pull/33605#issuecomment-891521262 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46487/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

AmplabJenkins commented on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891521264 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46482/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

AmplabJenkins commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-891521259 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46483/ --

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891520713 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46489/ -- This is an automated message from the Apache

[GitHub] [spark] wangyum commented on a change in pull request #33509: [SPARK-36280][SQL] Remove redundant aliases after RewritePredicateSubquery

wangyum commented on a change in pull request #33509: URL: https://github.com/apache/spark/pull/33509#discussion_r681429815 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SubquerySuite.scala ## @@ -1876,4 +1877,29 @@ class SubquerySuite extends QueryTest with

[GitHub] [spark] ulysses-you commented on pull request #32355: [SPARK-35221][SQL] Add join hint build side check

ulysses-you commented on pull request #32355: URL: https://github.com/apache/spark/pull/32355#issuecomment-891519254 thank you @cloud-fan for review, added two methods in `JoinSelection`: * `checkHintBuildSide` is to check hint build side * `checkHintNonEquiJoin` is to check hint equi

[GitHub] [spark] ulysses-you commented on a change in pull request #32355: [SPARK-35221][SQL] Add join hint build side check

ulysses-you commented on a change in pull request #32355: URL: https://github.com/apache/spark/pull/32355#discussion_r681429032 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/HintErrorLogger.scala ## @@ -42,6 +45,17 @@ object HintErrorLogger

[GitHub] [spark] SparkQA removed a comment on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA removed a comment on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891474270 **[Test build #141974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141974/testReport)** for PR 33583 at commit

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891518865 **[Test build #141974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141974/testReport)** for PR 33583 at commit

[GitHub] [spark] SparkQA commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

SparkQA commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891517771 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46488/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is received

SparkQA commented on pull request #33605: URL: https://github.com/apache/spark/pull/33605#issuecomment-891517354 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46487/ --

[GitHub] [spark] SparkQA commented on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

SparkQA commented on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891515597 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46482/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-891514642 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46483/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891508250 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46485/ -- This is an automated message from the

[GitHub] [spark] Peng-Lei commented on pull request #33618: [SPARK-36381][SQL] Add case sensitive and case insensitive compare for checking column name exist when alter table

Peng-Lei commented on pull request #33618: URL: https://github.com/apache/spark/pull/33618#issuecomment-891507272 @imback82 @cloud-fan Could you take a look ? I'm not quite sure if this is needed. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] Peng-Lei opened a new pull request #33618: [SPARK-36381][SQL] Add case sensitive and case insensitive compare for checking column name exist when alter table

Peng-Lei opened a new pull request #33618: URL: https://github.com/apache/spark/pull/33618 ### What changes were proposed in this pull request? Add the Resolver to `checkColumnNotExists` to check name exist in case sensitive. ### Why are the changes needed? At now the resolver

[GitHub] [spark] karenfeng commented on a change in pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

karenfeng commented on a change in pull request #33560: URL: https://github.com/apache/spark/pull/33560#discussion_r681416650 ## File path: core/src/main/resources/error/README.md ## @@ -79,16 +79,177 @@ The message format accepts string parameters via the C-style printf

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891501800 **[Test build #141978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141978/testReport)** for PR 33583 at commit

[GitHub] [spark] SparkQA commented on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

SparkQA commented on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891499615 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46486/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

SparkQA commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891499166 **[Test build #141977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141977/testReport)** for PR 33616 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

AmplabJenkins removed a comment on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891497684 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] dongjoon-hyun commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

dongjoon-hyun commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891498429 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

AmplabJenkins removed a comment on pull request #33560: URL: https://github.com/apache/spark/pull/33560#issuecomment-891497592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141968/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33590: [SPARK-36359][SQL] Coalesce drop all expressions after the first non nullable expression

AmplabJenkins removed a comment on pull request #33590: URL: https://github.com/apache/spark/pull/33590#issuecomment-891497591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46480/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33587: [SPARK-36355][SQL] NamedExpression add method `withName(newName: String)

AmplabJenkins removed a comment on pull request #33587: URL: https://github.com/apache/spark/pull/33587#issuecomment-891497593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46484/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

AmplabJenkins removed a comment on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891497590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141975/

[GitHub] [spark] SparkQA commented on pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is received

SparkQA commented on pull request #33605: URL: https://github.com/apache/spark/pull/33605#issuecomment-891497830 **[Test build #141976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141976/testReport)** for PR 33605 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

AmplabJenkins removed a comment on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891497594 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141971/

[GitHub] [spark] AmplabJenkins commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

AmplabJenkins commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891497684 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

AmplabJenkins commented on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891497590 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141975/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error m…

AmplabJenkins commented on pull request #33617: URL: https://github.com/apache/spark/pull/33617#issuecomment-891497668 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

AmplabJenkins commented on pull request #33560: URL: https://github.com/apache/spark/pull/33560#issuecomment-891497592 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141968/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33587: [SPARK-36355][SQL] NamedExpression add method `withName(newName: String)

AmplabJenkins commented on pull request #33587: URL: https://github.com/apache/spark/pull/33587#issuecomment-891497593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46484/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33590: [SPARK-36359][SQL] Coalesce drop all expressions after the first non nullable expression

AmplabJenkins commented on pull request #33590: URL: https://github.com/apache/spark/pull/33590#issuecomment-891497591 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46480/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

AmplabJenkins commented on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891497594 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141971/ -- This

[GitHub] [spark] zhuqi-lucas commented on pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error m…

zhuqi-lucas commented on pull request #33617: URL: https://github.com/apache/spark/pull/33617#issuecomment-891496800 cc @Ngone51 @zhouyejoe @mridulm @HyukjinKwon Could you help review this, thanks. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] venkata91 commented on a change in pull request #33613: [SPARK-36378][SHUFFLE] Minor changes to address a few identified push based shuffle server side inefficiencies.

venkata91 commented on a change in pull request #33613: URL: https://github.com/apache/spark/pull/33613#discussion_r681410950 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -472,8 +493,8 @@ public void

[GitHub] [spark] ReachInfi commented on pull request #33314: [SPARK-36118][SQL] Add bitmap functions for Spark SQL

ReachInfi commented on pull request #33314: URL: https://github.com/apache/spark/pull/33314#issuecomment-891495888 Hi, do i need update function.scala and FunctionRegistry.scala? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun commented on pull request #33614: [SPARK-36367][3.2][PYTHON] Partially backport to avoid unexpected error with pandas 1.3

dongjoon-hyun commented on pull request #33614: URL: https://github.com/apache/spark/pull/33614#issuecomment-891495714 Do we need to some documentation about Pandas 1.3 for Apache Spark 3.2? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] zhuqi-lucas opened a new pull request #33617: [SPARK-35548][CORE][SHUFFLE] Handling new attempt has started error m…

zhuqi-lucas opened a new pull request #33617: URL: https://github.com/apache/spark/pull/33617 …essage in BlockPushErrorHandler in client. ### What changes were proposed in this pull request? Add a new type of error message in BlockPushErrorHandler which indicates the

[GitHub] [spark] SparkQA commented on pull request #33587: [SPARK-36355][SQL] NamedExpression add method `withName(newName: String)

SparkQA commented on pull request #33587: URL: https://github.com/apache/spark/pull/33587#issuecomment-891495164 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46484/ --

[GitHub] [spark] venkata91 commented on a change in pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is receive

venkata91 commented on a change in pull request #33605: URL: https://github.com/apache/spark/pull/33605#discussion_r681409675 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -1234,6 +1234,26 @@ void

[GitHub] [spark] SparkQA commented on pull request #33583: [WIP][SPARK-36352][SQL] Spark should check result plan's output schema name

SparkQA commented on pull request #33583: URL: https://github.com/apache/spark/pull/33583#issuecomment-891494235 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46485/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

SparkQA commented on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891494086 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46482/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-891493559 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46483/ -- This is an automated message from the Apache

[GitHub] [spark] beliefer commented on pull request #33520: [SPARK-36289][SQL] Rewrite distinct count case when expressions without Expand node

beliefer commented on pull request #33520: URL: https://github.com/apache/spark/pull/33520#issuecomment-891492579 > > What is `cond2`? > > @beliefer `cond1` and `cond2` are two conditions such as ` a > 1` It seems you should correct the description. So reviewer could

[GitHub] [spark] venkata91 commented on a change in pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is receive

venkata91 commented on a change in pull request #33605: URL: https://github.com/apache/spark/pull/33605#discussion_r681407159 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -1234,6 +1234,26 @@ void

[GitHub] [spark] venkata91 commented on a change in pull request #33605: [SPARK-32923][FOLLOW-UP] Clean up older shuffleMergeId shuffle files when finalize request for higher shuffleMergeId is receive

venkata91 commented on a change in pull request #33605: URL: https://github.com/apache/spark/pull/33605#discussion_r681406589 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -1234,6 +1234,26 @@ void

[GitHub] [spark] SparkQA removed a comment on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

SparkQA removed a comment on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891474191 **[Test build #141971 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141971/testReport)** for PR 33607 at commit

[GitHub] [spark] SparkQA commented on pull request #33607: [SPARK-36175][SQL][FOLLOWUP] Improve the comments for AvroDeserializer/AvroSerializer

SparkQA commented on pull request #33607: URL: https://github.com/apache/spark/pull/33607#issuecomment-891487921 **[Test build #141971 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141971/testReport)** for PR 33607 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

SparkQA removed a comment on pull request #33560: URL: https://github.com/apache/spark/pull/33560#issuecomment-891426186 **[Test build #141968 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141968/testReport)** for PR 33560 at commit

[GitHub] [spark] SparkQA commented on pull request #33560: [SPARK-36331][CORE] Add standard SQLSTATEs to error guidelines

SparkQA commented on pull request #33560: URL: https://github.com/apache/spark/pull/33560#issuecomment-891485688 **[Test build #141968 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141968/testReport)** for PR 33560 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33608: [SPARK-36379][SQL] Null at root level of a JSON array should not fail w/ permissive mode

dongjoon-hyun commented on pull request #33608: URL: https://github.com/apache/spark/pull/33608#issuecomment-891484659 It's only about backporting to branch-3.1~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] otterc commented on pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

otterc commented on pull request #33616: URL: https://github.com/apache/spark/pull/33616#issuecomment-891484143 @mridulm @Ngone51 @Victsm @dongjoon-hyun @zhouyejoe @venkata91 Please take a look -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] otterc opened a new pull request #33616: [SPARK-36389][CORE][SHUFFLE] Revert the change that accepts negative mapId in ShuffleBlockId

otterc opened a new pull request #33616: URL: https://github.com/apache/spark/pull/33616 ### What changes were proposed in this pull request? With SPARK-32922, we added a change that ShuffleBlockId can have a negative mapId. This was to support push-based shuffle where -1 as mapId

[GitHub] [spark] SparkQA removed a comment on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

SparkQA removed a comment on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891477274 **[Test build #141975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141975/testReport)** for PR 33615 at commit

[GitHub] [spark] SparkQA commented on pull request #33615: [SPARK-36374][SHUFFLE][DOC] Push-based shuffle high level user documentation

SparkQA commented on pull request #33615: URL: https://github.com/apache/spark/pull/33615#issuecomment-891483233 **[Test build #141975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141975/testReport)** for PR 33615 at commit

  1   2   3   4   5   6   >