[GitHub] [spark] ReachInfi commented on pull request #33314: Add bitmap functions in Spark SQL

ReachInfi commented on pull request #33314: URL: https://github.com/apache/spark/pull/33314#issuecomment-878802086 https://issues.apache.org/jira/browse/SPARK-36118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] wangyum commented on pull request #33286: [SPARK-36079][SQL] Null-based filter estimate should always be in the range [0, 1]

wangyum commented on pull request #33286: URL: https://github.com/apache/spark/pull/33286#issuecomment-878801153 retest this please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

SparkQA removed a comment on pull request #33311: URL: https://github.com/apache/spark/pull/33311#issuecomment-878708461 **[Test build #140946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140946/testReport)** for PR 33311 at commit

[GitHub] [spark] SparkQA commented on pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

SparkQA commented on pull request #33311: URL: https://github.com/apache/spark/pull/33311#issuecomment-878797982 **[Test build #140946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140946/testReport)** for PR 33311 at commit

[GitHub] [spark] ReachInfi commented on pull request #33314: Add bitmap functions in Spark SQL

ReachInfi commented on pull request #33314: URL: https://github.com/apache/spark/pull/33314#issuecomment-878797178 Ok,tks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] viirya commented on a change in pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

viirya commented on a change in pull request #33311: URL: https://github.com/apache/spark/pull/33311#discussion_r668442575 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala ## @@ -2429,8 +2429,9 @@ class

[GitHub] [spark] viirya commented on a change in pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

viirya commented on a change in pull request #33311: URL: https://github.com/apache/spark/pull/33311#discussion_r668442575 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala ## @@ -2429,8 +2429,9 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

HeartSaVioR commented on a change in pull request #33311: URL: https://github.com/apache/spark/pull/33311#discussion_r668441780 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala ## @@ -2429,8 +2429,9 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins removed a comment on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878794471 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140953/

[GitHub] [spark] HeartSaVioR commented on a change in pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

HeartSaVioR commented on a change in pull request #33311: URL: https://github.com/apache/spark/pull/33311#discussion_r668441137 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/StreamTest.scala ## @@ -871,6 +871,9 @@ trait StreamTest extends QueryTest with

[GitHub] [spark] AmplabJenkins commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878794471 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140953/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA removed a comment on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878749337 **[Test build #140953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140953/testReport)** for PR 33258 at commit

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878794174 **[Test build #140953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140953/testReport)** for PR 33258 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

AmplabJenkins removed a comment on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878789756 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better

SparkQA commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-878790692 **[Test build #140956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140956/testReport)** for PR 33078 at commit

[GitHub] [spark] SparkQA commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

SparkQA commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878790553 **[Test build #140955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140955/testReport)** for PR 33310 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins removed a comment on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878789751 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45467/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

AmplabJenkins removed a comment on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878789749 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45466/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins removed a comment on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878789750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45468/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

AmplabJenkins removed a comment on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878789748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140947/

[GitHub] [spark] AmplabJenkins commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878789751 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45467/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

AmplabJenkins commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878789749 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45466/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

AmplabJenkins commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878789748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140947/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

AmplabJenkins commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878789756 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878789750 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45468/ --

[GitHub] [spark] HyukjinKwon commented on pull request #33314: Add bitmap functions in Spark SQL

HyukjinKwon commented on pull request #33314: URL: https://github.com/apache/spark/pull/33314#issuecomment-878784758 @ReachInfi can you file a JIRA and link it to the PR title please? See also https://spark.apache.org/contributing.html -- This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

dongjoon-hyun commented on a change in pull request #33311: URL: https://github.com/apache/spark/pull/33311#discussion_r668430347 ## File path: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaMicroBatchSourceSuite.scala ## @@ -2429,8 +2429,9 @@ class

[GitHub] [spark] dongjoon-hyun commented on pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

dongjoon-hyun commented on pull request #33311: URL: https://github.com/apache/spark/pull/33311#issuecomment-878782539 Thank you, @viirya . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

dongjoon-hyun commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878781192 +1, LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] allisonwang-db commented on a change in pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

allisonwang-db commented on a change in pull request #33070: URL: https://github.com/apache/spark/pull/33070#discussion_r668428086 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/DecorrelateInnerQuery.scala ## @@ -428,7 +451,132 @@ object

[GitHub] [spark] michaelzhang-db removed a comment on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

michaelzhang-db removed a comment on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878780180 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA removed a comment on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

SparkQA removed a comment on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878728629 **[Test build #140947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140947/testReport)** for PR 33312 at commit

[GitHub] [spark] michaelzhang-db commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

michaelzhang-db commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878780180 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

SparkQA commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878779979 **[Test build #140947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140947/testReport)** for PR 33312 at commit

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-87895 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45467/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #33299: [SPARK-36046][SQL] Support new functions make_timestamp_ntz and make_timestamp_ltz

cloud-fan commented on a change in pull request #33299: URL: https://github.com/apache/spark/pull/33299#discussion_r668423810 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala ## @@ -552,6 +552,8 @@ object FunctionRegistry

[GitHub] [spark] allisonwang-db commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

allisonwang-db commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-878776521 > Next we should use this fix to solve the count bug for all correlated subqueries. Created two follow-up issues: SPARK-36113 and SPARK-36115. -- This is an

[GitHub] [spark] SparkQA commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

SparkQA commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878775838 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45466/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878775341 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45468/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33314: Add bitmap functions in Spark SQL

AmplabJenkins commented on pull request #33314: URL: https://github.com/apache/spark/pull/33314#issuecomment-878767184 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

AmplabJenkins removed a comment on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878767004 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45465/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

AmplabJenkins removed a comment on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878767002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45464/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

AmplabJenkins removed a comment on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878767000 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45463/

[GitHub] [spark] AmplabJenkins commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

AmplabJenkins commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878767000 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45463/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

AmplabJenkins commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878767002 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45464/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

AmplabJenkins commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878767004 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45465/ --

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878764568 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45467/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon closed pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

HyukjinKwon closed pull request #33312: URL: https://github.com/apache/spark/pull/33312 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon closed pull request #33315: [SPARK-36104][PYTHON][FOLLOWUP] Remove unused import "typing.cast"

HyukjinKwon closed pull request #33315: URL: https://github.com/apache/spark/pull/33315 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

HyukjinKwon commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878763360 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on pull request #33315: [SPARK-36104][PYTHON][FOLLOWUP] Remove unused import "typing.cast"

HyukjinKwon commented on pull request #33315: URL: https://github.com/apache/spark/pull/33315#issuecomment-878763205 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

SparkQA commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878762536 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45466/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878762434 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45468/ -- This is an automated message from the Apache

[GitHub] [spark] sarutak commented on pull request #33315: [SPARK-36104][PYTHON][FOLLOWUP] Remove unused import "typing.cast"

sarutak commented on pull request #33315: URL: https://github.com/apache/spark/pull/33315#issuecomment-878762244 cc: @HyukjinKwon @xinrong-databricks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] sarutak opened a new pull request #33315: [SPARK-36104][PYTHON][FOLLOWUP] Remove unused import "typing.cast"

sarutak opened a new pull request #33315: URL: https://github.com/apache/spark/pull/33315 ### What changes were proposed in this pull request? This is a followup PR for SPARK-36104 (#33307) and removes unused import `typing.cast`. ### Why are the changes needed? To

[GitHub] [spark] SparkQA commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

SparkQA commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878761359 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45465/ --

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r668409304 ## File path: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ## @@ -360,13 +389,41 @@ private[spark] class

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r668408590 ## File path: core/src/main/java/org/apache/spark/shuffle/checksum/ShuffleChecksumHelper.java ## @@ -0,0 +1,81 @@ +package

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r668408329 ## File path: core/src/main/scala/org/apache/spark/storage/DiskBlockObjectWriter.scala ## @@ -101,12 +108,30 @@ private[spark] class

[GitHub] [spark] ReachInfi opened a new pull request #33314: Add bitmap functions in Spark SQL

ReachInfi opened a new pull request #33314: URL: https://github.com/apache/spark/pull/33314 ### What changes were proposed in this pull request? add functions of bitmap building and computing cardinality for Spark SQL, If this is ok, I will update function.scala and

[GitHub] [spark] dgd-contributor commented on a change in pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

dgd-contributor commented on a change in pull request #33293: URL: https://github.com/apache/spark/pull/33293#discussion_r668407288 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -344,29 +344,28 @@ object DateTimeUtils {

[GitHub] [spark] SparkQA commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

SparkQA commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878756758 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45463/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

SparkQA commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878754781 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45464/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

AmplabJenkins removed a comment on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878753027 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45460/

[GitHub] [spark] AmplabJenkins commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

AmplabJenkins commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878753027 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45460/ --

[GitHub] [spark] SparkQA commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

SparkQA commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878753012 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45460/ -- This is an automated message from the

[GitHub] [spark] beliefer commented on a change in pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

beliefer commented on a change in pull request #33258: URL: https://github.com/apache/spark/pull/33258#discussion_r668401909 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -236,6 +274,8 @@ case class

[GitHub] [spark] SparkQA removed a comment on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA removed a comment on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878749770 **[Test build #140954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140954/testReport)** for PR 30869 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

AmplabJenkins removed a comment on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878685125 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins removed a comment on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878750245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140954/

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r668400943 ## File path: core/src/main/scala/org/apache/spark/shuffle/IndexShuffleBlockResolver.scala ## @@ -360,13 +389,41 @@ private[spark] class

[GitHub] [spark] AmplabJenkins commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

AmplabJenkins commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878750245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140954/ -- This

[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878750232 **[Test build #140954 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140954/testReport)** for PR 30869 at commit

[GitHub] [spark] SparkQA commented on pull request #30869: [SPARK-33865][SQL] When HiveDDL, we need check avro schema too

SparkQA commented on pull request #30869: URL: https://github.com/apache/spark/pull/30869#issuecomment-878749770 **[Test build #140954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140954/testReport)** for PR 30869 at commit

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-878749337 **[Test build #140953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140953/testReport)** for PR 33258 at commit

[GitHub] [spark] SparkQA commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

SparkQA commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878749275 **[Test build #140951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140951/testReport)** for PR 33310 at commit

[GitHub] [spark] SparkQA commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

SparkQA commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878749259 **[Test build #140952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140952/testReport)** for PR 33297 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

AmplabJenkins removed a comment on pull request #33311: URL: https://github.com/apache/spark/pull/33311#issuecomment-878748105 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45459/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33287: [SPARK-36081][SPARK-36066][SQL] Update the document about the behavior change of trimming characters for cast

AmplabJenkins removed a comment on pull request #33287: URL: https://github.com/apache/spark/pull/33287#issuecomment-878748106 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140945/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32949: [SPARK-35749][SPARK-35773][SQL] Parse unit list interval literals as tightest year-month/day-time interval types

AmplabJenkins removed a comment on pull request #32949: URL: https://github.com/apache/spark/pull/32949#issuecomment-878748107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140939/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

AmplabJenkins removed a comment on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878748108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45462/

[GitHub] [spark] AmplabJenkins commented on pull request #32949: [SPARK-35749][SPARK-35773][SQL] Parse unit list interval literals as tightest year-month/day-time interval types

AmplabJenkins commented on pull request #32949: URL: https://github.com/apache/spark/pull/32949#issuecomment-878748107 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140939/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33287: [SPARK-36081][SPARK-36066][SQL] Update the document about the behavior change of trimming characters for cast

AmplabJenkins commented on pull request #33287: URL: https://github.com/apache/spark/pull/33287#issuecomment-878748106 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140945/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

AmplabJenkins commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878748108 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45462/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33311: [SPARK-36109][SS][TEST] Check data after adding data to topic in KafkaSourceStressSuite

AmplabJenkins commented on pull request #33311: URL: https://github.com/apache/spark/pull/33311#issuecomment-878748105 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45459/ --

[GitHub] [spark] SparkQA commented on pull request #33312: [SPARK-36110][BUILD] Upgrade SBT to 1.5.5

SparkQA commented on pull request #33312: URL: https://github.com/apache/spark/pull/33312#issuecomment-878747011 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45462/ --

[GitHub] [spark] SparkQA commented on pull request #33297: [SPARK-36069] from_json's exception should contain field name, type and value

SparkQA commented on pull request #33297: URL: https://github.com/apache/spark/pull/33297#issuecomment-878746639 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45463/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

SparkQA commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878744867 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45464/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon closed pull request #33302: Revert "[SPARK-35253][SPARK-35398][SQL][BUILD] Bump up the janino version to v3.1.4"

HyukjinKwon closed pull request #33302: URL: https://github.com/apache/spark/pull/33302 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33302: Revert "[SPARK-35253][SPARK-35398][SQL][BUILD] Bump up the janino version to v3.1.4"

HyukjinKwon commented on pull request #33302: URL: https://github.com/apache/spark/pull/33302#issuecomment-878744589 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

HyukjinKwon commented on a change in pull request #33310: URL: https://github.com/apache/spark/pull/33310#discussion_r668394926 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/ShuffledRowRDD.scala ## @@ -58,6 +58,11 @@ case class

[GitHub] [spark] HyukjinKwon commented on pull request #33310: [WIP][SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

HyukjinKwon commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-878743794 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #33287: [SPARK-36081][SPARK-36066][SQL] Update the document about the behavior change of trimming characters for cast

SparkQA removed a comment on pull request #33287: URL: https://github.com/apache/spark/pull/33287#issuecomment-878687971 **[Test build #140945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140945/testReport)** for PR 33287 at commit

[GitHub] [spark] SparkQA commented on pull request #33287: [SPARK-36081][SPARK-36066][SQL] Update the document about the behavior change of trimming characters for cast

SparkQA commented on pull request #33287: URL: https://github.com/apache/spark/pull/33287#issuecomment-878743114 **[Test build #140945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140945/testReport)** for PR 33287 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

HyukjinKwon commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878742789 cc @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

HyukjinKwon commented on a change in pull request #33293: URL: https://github.com/apache/spark/pull/33293#discussion_r668394064 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -344,29 +344,28 @@ object DateTimeUtils {

[GitHub] [spark] HyukjinKwon closed pull request #33307: [SPARK-36104][PYTHON] Manage InternalField in DataTypeOps.neg/abs

HyukjinKwon closed pull request #33307: URL: https://github.com/apache/spark/pull/33307 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33307: [SPARK-36104][PYTHON] Manage InternalField in DataTypeOps.neg/abs

HyukjinKwon commented on pull request #33307: URL: https://github.com/apache/spark/pull/33307#issuecomment-878741636 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

SparkQA commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878741530 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45460/ -- This is an automated message from the Apache

[GitHub] [spark] dgd-contributor commented on a change in pull request #33291: [SPARK-35561][SQL] Remove leading zeros from empty static number type partition

dgd-contributor commented on a change in pull request #33291: URL: https://github.com/apache/spark/pull/33291#discussion_r668391894 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala ## @@ -351,10 +351,24 @@ object

  1   2   3   4   5   6   >