[GitHub] [spark] sarutak opened a new pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
sarutak opened a new pull request #34649: URL: https://github.com/apache/spark/pull/34649 ### What changes were proposed in this pull request? This PR mitigate an issue that MiMa fails for Scala 2.13 after SPARK-35672 (##34120). ``` $ dev/change-scala-version.sh 2.13 $

[GitHub] [spark] attilapiros commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
attilapiros commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973068166 Thanks @sarutak for taking care of this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] mridulm commented on a change in pull request #34632: [SPARK-37356][CORE] Add fine grained locking to the BlockInfoManager

2021-11-18 Thread GitBox
mridulm commented on a change in pull request #34632: URL: https://github.com/apache/spark/pull/34632#discussion_r752476094 ## File path: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala ## @@ -166,6 +189,48 @@ private[storage] class BlockInfoManager extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973101827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145388/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973101824 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145395/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973148314 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145400/

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973166396 **[Test build #145402 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145402/testReport)** for PR 34650 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973156390 **[Test build #145402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145402/testReport)** for PR 34650 at commit

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973193881 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49875/ -- This is an automated message from the Apache

[GitHub] [spark] HeartSaVioR commented on a change in pull request #34502: [SPARK-37224][SS] Optimize write path on RocksDB state store provider

2021-11-18 Thread GitBox
HeartSaVioR commented on a change in pull request #34502: URL: https://github.com/apache/spark/pull/34502#discussion_r752576654 ## File path: docs/structured-streaming-programming-guide.md ## @@ -1956,8 +1956,21 @@ Here are the configs regarding to RocksDB instance of the

[GitHub] [spark] SparkQA commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973214542 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49874/ -- This is an automated message from the

[GitHub] [spark] kazuyukitanimura commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
kazuyukitanimura commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r752601914 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,46 @@

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973244493 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49876/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973244532 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49876/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973244532 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49876/

[GitHub] [spark] mridulm commented on a change in pull request #34632: [SPARK-37356][CORE] Add fine grained locking to the BlockInfoManager

2021-11-18 Thread GitBox
mridulm commented on a change in pull request #34632: URL: https://github.com/apache/spark/pull/34632#discussion_r752507849 ## File path: core/src/main/scala/org/apache/spark/storage/BlockInfoManager.scala ## @@ -341,106 +415,103 @@ private[storage] class BlockInfoManager

[GitHub] [spark] SparkQA removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973114717 **[Test build #145400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145400/testReport)** for PR 34636 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973148314 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145400/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973194334 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973194329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973194331 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49871/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973194331 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49871/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973202711 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49873/

[GitHub] [spark] AmplabJenkins commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973202711 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49873/ --

[GitHub] [spark] SparkQA commented on pull request #34593: [SPARK-37324][SQL] Adds support for decimal rounding mode up, down, half_down

2021-11-18 Thread GitBox
SparkQA commented on pull request #34593: URL: https://github.com/apache/spark/pull/34593#issuecomment-973244136 **[Test build #145406 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145406/testReport)** for PR 34593 at commit

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

2021-11-18 Thread GitBox
SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-973244430 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49878/ -- This is an automated message from the Apache

[GitHub] [spark] HeartSaVioR commented on pull request #34652: [SPARK-37224][SS][FOLLOWUP] Clarify the guide doc and fix the method doc

2021-11-18 Thread GitBox
HeartSaVioR commented on pull request #34652: URL: https://github.com/apache/spark/pull/34652#issuecomment-973265193 cc. @viirya @xuanyuanking -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HeartSaVioR opened a new pull request #34652: [SPARK-37224][SS][FOLLOWUP] Clarify the guide doc and fix the method doc

2021-11-18 Thread GitBox
HeartSaVioR opened a new pull request #34652: URL: https://github.com/apache/spark/pull/34652 ### What changes were proposed in this pull request? This PR is a follow-up of #34502 to address post-reviews. This PR rewords on the explanation on performance tune on RocksDB state

[GitHub] [spark] SparkQA removed a comment on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973150233 **[Test build #145401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145401/testReport)** for PR 34651 at commit

[GitHub] [spark] SparkQA commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

2021-11-18 Thread GitBox
SparkQA commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-973017066 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49869/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp with local time zone for orc

2021-11-18 Thread GitBox
SparkQA commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973035927 **[Test build #145385 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145385/testReport)** for PR 34644 at commit

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973062089 **[Test build #145388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145388/testReport)** for PR 34611 at commit

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973114717 **[Test build #145400 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145400/testReport)** for PR 34636 at commit

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973162274 **[Test build #145403 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145403/testReport)** for PR 34650 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973241342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49875/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973241347 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145398/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973241347 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145398/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973241345 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49874/

[GitHub] [spark] hvanhovell closed pull request #34632: [SPARK-37356][CORE] Add fine grained locking to the BlockInfoManager

2021-11-18 Thread GitBox
hvanhovell closed pull request #34632: URL: https://github.com/apache/spark/pull/34632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34645: [SPARK-37371][SQL] UnionExec should support columnar if all children support columnar

2021-11-18 Thread GitBox
SparkQA commented on pull request #34645: URL: https://github.com/apache/spark/pull/34645#issuecomment-973292081 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49880/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973291905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145403/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973291906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49877/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973291906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49877/

[GitHub] [spark] AmplabJenkins commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-973291903 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49878/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973291904 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145401/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973291905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145403/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-973291903 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49878/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973291904 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145401/

[GitHub] [spark] SparkQA commented on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
SparkQA commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973034670 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49868/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973125812 **[Test build #145400 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145400/testReport)** for PR 34636 at commit

[GitHub] [spark] attilapiros opened a new pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
attilapiros opened a new pull request #34651: URL: https://github.com/apache/spark/pull/34651 ### What changes were proposed in this pull request? Collecting `LocalSparkContext` worker logs in case of test failure. ### Why are the changes needed? About 50 test

[GitHub] [spark] sunchao commented on pull request #34613: [SPARK-37342][BUILD] Upgrade Apache Arrow to 6.0.0

2021-11-18 Thread GitBox
sunchao commented on pull request #34613: URL: https://github.com/apache/spark/pull/34613#issuecomment-973153680 Hmm I must missed something since I always get this "ConnectionRefusedError: [Errno 61] Connection refused" even though I followed the exact above steps. Not sure what host &

[GitHub] [spark] kazuyukitanimura commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
kazuyukitanimura commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r752537385 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OnHeapColumnVector.java ## @@ -147,6 +147,18 @@ public void

[GitHub] [spark] sunchao commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
sunchao commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r752544556 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,46 @@ public

[GitHub] [spark] SparkQA commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973179932 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49874/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973209654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145404/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973209654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145404/

[GitHub] [spark] SparkQA removed a comment on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973196575 **[Test build #145404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145404/testReport)** for PR 34636 at commit

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973209346 **[Test build #145404 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145404/testReport)** for PR 34636 at commit

[GitHub] [spark] SparkQA commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
SparkQA commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973229564 **[Test build #145398 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145398/testReport)** for PR 34649 at commit

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973230438 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49877/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973229794 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49875/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973101987 **[Test build #145398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145398/testReport)** for PR 34649 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973241342 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49875/

[GitHub] [spark] SparkQA commented on pull request #32875: [SPARK-35703][SQL] Relax constraint for bucket join and remove HashClusteredDistribution

2021-11-18 Thread GitBox
SparkQA commented on pull request #32875: URL: https://github.com/apache/spark/pull/32875#issuecomment-973242337 **[Test build #145405 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145405/testReport)** for PR 32875 at commit

[GitHub] [spark] SparkQA commented on pull request #34645: [SPARK-37371][SQL] UnionExec should support columnar if all children support columnar

2021-11-18 Thread GitBox
SparkQA commented on pull request #34645: URL: https://github.com/apache/spark/pull/34645#issuecomment-973259922 **[Test build #145407 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145407/testReport)** for PR 34645 at commit

[GitHub] [spark] viirya commented on pull request #34645: [SPARK-37371][SQL] UnionExec should support columnar if all children support columnar

2021-11-18 Thread GitBox
viirya commented on pull request #34645: URL: https://github.com/apache/spark/pull/34645#issuecomment-973260082 cc @cloud-fan @sunchao @dongjoon-hyun -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973269669 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49877/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34645: [SPARK-37371][SQL] UnionExec should support columnar if all children support columnar

2021-11-18 Thread GitBox
SparkQA commented on pull request #34645: URL: https://github.com/apache/spark/pull/34645#issuecomment-973293622 **[Test build #145409 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145409/testReport)** for PR 34645 at commit

[GitHub] [spark] SparkQA commented on pull request #34593: [SPARK-37324][SQL] Adds support for decimal rounding mode up, down, half_down

2021-11-18 Thread GitBox
SparkQA commented on pull request #34593: URL: https://github.com/apache/spark/pull/34593#issuecomment-973293666 **[Test build #145410 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145410/testReport)** for PR 34593 at commit

[GitHub] [spark] sunchao commented on a change in pull request #34645: [SPARK-37371][SQL] UnionExec should support columnar if all children support columnar

2021-11-18 Thread GitBox
sunchao commented on a change in pull request #34645: URL: https://github.com/apache/spark/pull/34645#discussion_r752645500 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala ## @@ -1355,6 +1357,45 @@ class DataFrameSetOperationsSuite

[GitHub] [spark] AmplabJenkins commented on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973037748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49868/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-973037746 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49869/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp with local time zone for orc

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973037789 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145385/

[GitHub] [spark] AmplabJenkins commented on pull request #34634: [SPARK-37357][SQL] Create skew partition specs should respect min partition size

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34634: URL: https://github.com/apache/spark/pull/34634#issuecomment-973037747 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49870/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973037748 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49868/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34634: [SPARK-37357][SQL] Create skew partition specs should respect min partition size

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34634: URL: https://github.com/apache/spark/pull/34634#issuecomment-973037747 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49870/

[GitHub] [spark] AmplabJenkins commented on pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34647: URL: https://github.com/apache/spark/pull/34647#issuecomment-973037746 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49869/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-973101827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145388/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973101824 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145395/ -- This

[GitHub] [spark] SparkQA commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
SparkQA commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973101987 **[Test build #145398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145398/testReport)** for PR 34649 at commit

[GitHub] [spark] SparkQA commented on pull request #34634: [SPARK-37357][SQL] Create skew partition specs should respect min partition size

2021-11-18 Thread GitBox
SparkQA commented on pull request #34634: URL: https://github.com/apache/spark/pull/34634#issuecomment-973200417 **[Test build #145397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145397/testReport)** for PR 34634 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973241345 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49874/ --

[GitHub] [spark] SparkQA commented on pull request #34634: [SPARK-37357][SQL] Create skew partition specs should respect min partition size

2021-11-18 Thread GitBox
SparkQA commented on pull request #34634: URL: https://github.com/apache/spark/pull/34634#issuecomment-973034173 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49870/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp with local time zone for orc

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-973037789 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145385/ -- This

[GitHub] [spark] SparkQA commented on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
SparkQA commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-973084694 **[Test build #145395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145395/testReport)** for PR 34326 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

2021-11-18 Thread GitBox
SparkQA removed a comment on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-972924514 **[Test build #145395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145395/testReport)** for PR 34326 at commit

[GitHub] [spark] holdenk opened a new pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
holdenk opened a new pull request #34650: URL: https://github.com/apache/spark/pull/34650 ### What changes were proposed in this pull request? Keep track of and communicate to the listener bus how long we are waiting for execs to be allocated from the underlying cluster manager.

[GitHub] [spark] SparkQA commented on pull request #34649: [SPARK-35672][FOLLOWUP][TESTS] Add an exclusion rule to MimaExcludes.scala for Scala 2.13.

2021-11-18 Thread GitBox
SparkQA commented on pull request #34649: URL: https://github.com/apache/spark/pull/34649#issuecomment-973136239 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49871/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973144066 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49872/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973150233 **[Test build #145401 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145401/testReport)** for PR 34651 at commit

[GitHub] [spark] sunchao commented on a change in pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

2021-11-18 Thread GitBox
sunchao commented on a change in pull request #34611: URL: https://github.com/apache/spark/pull/34611#discussion_r752516781 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedPlainValuesReader.java ## @@ -53,19 +53,46 @@ public

[GitHub] [spark] SparkQA commented on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
SparkQA commented on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973156390 **[Test build #145402 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145402/testReport)** for PR 34650 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34352: [SPARK-37018][SQL] Spark SQL should support create function with Aggregator

2021-11-18 Thread GitBox
AmplabJenkins commented on pull request #34352: URL: https://github.com/apache/spark/pull/34352#issuecomment-973156024 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145393/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34352: [SPARK-37018][SQL] Spark SQL should support create function with Aggregator

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34352: URL: https://github.com/apache/spark/pull/34352#issuecomment-973156024 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145393/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34650: [WIP][SPARK-36664][CORE] Log time waiting for cluster resources

2021-11-18 Thread GitBox
AmplabJenkins removed a comment on pull request #34650: URL: https://github.com/apache/spark/pull/34650#issuecomment-973118297 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145399/

[GitHub] [spark] SparkQA commented on pull request #34636: [WIP][SPARK-37359][K8S] Cleanup the Spark Kubernetes Integration tests

2021-11-18 Thread GitBox
SparkQA commented on pull request #34636: URL: https://github.com/apache/spark/pull/34636#issuecomment-973196575 **[Test build #145404 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145404/testReport)** for PR 34636 at commit

[GitHub] [spark] SparkQA commented on pull request #34651: [WIP][SPARK-37373] Collecting LocalSparkContext worker logs in case of test failure

2021-11-18 Thread GitBox
SparkQA commented on pull request #34651: URL: https://github.com/apache/spark/pull/34651#issuecomment-973277581 **[Test build #145401 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145401/testReport)** for PR 34651 at commit

[GitHub] [spark] SparkQA commented on pull request #34593: [SPARK-37324][SQL] Adds support for decimal rounding mode up, down, half_down

2021-11-18 Thread GitBox
SparkQA commented on pull request #34593: URL: https://github.com/apache/spark/pull/34593#issuecomment-973284096 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49879/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34652: [SPARK-37224][SS][FOLLOWUP] Clarify the guide doc and fix the method doc

2021-11-18 Thread GitBox
SparkQA commented on pull request #34652: URL: https://github.com/apache/spark/pull/34652#issuecomment-973293544 **[Test build #145408 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145408/testReport)** for PR 34652 at commit

  1   2   3   4   5   6   7   8   >