[GitHub] [spark] AmplabJenkins commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins

AmplabJenkins commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644476807 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins

SparkQA commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644476504 **[Test build #124081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124081/testReport)** for PR 28810 at commit [`1251b68`](https://github.com

[GitHub] [spark] maropu commented on pull request #28810: [SPARK-31705][SQL][FOLLOWUP] Avoid the unnecessary CNF computation for full-outer joins

maropu commented on pull request #28810: URL: https://github.com/apache/spark/pull/28810#issuecomment-644474897 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] HyukjinKwon commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

HyukjinKwon commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644475032 Let's also follow the new GitHub PR template - https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE ---

[GitHub] [spark] xuanyuanking edited a comment on pull request #28707: [SPARK-31894][SS] Introduce UnsafeRow format validation for streaming state store

xuanyuanking edited a comment on pull request #28707: URL: https://github.com/apache/spark/pull/28707#issuecomment-643916110 cc @maropu @gatorsmile @HeartSaVioR @dongjoon-hyun A new regression bug SPARK-31990 was found when investigating the test failure https://github.com/apache/sp

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644468103 I've updated the description @gatorsmile, let me know if there is any particular points you would like clarified. --

[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL] Enable codegen for GenerateExec

maropu commented on a change in pull request #28715: URL: https://github.com/apache/spark/pull/28715#discussion_r440526114 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala ## @@ -51,6 +51,93 @@ class WholeStageCodegenSuite exten

[GitHub] [spark] maropu commented on pull request #28715: [SPARK-31897][SQL]Enable codegen for GenerateExec

maropu commented on pull request #28715: URL: https://github.com/apache/spark/pull/28715#issuecomment-644466372 Could you show us performance gains by enabling this codegen? This is an automated message from the Apache Git Se

[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL]Enable codegen for GenerateExec

maropu commented on a change in pull request #28715: URL: https://github.com/apache/spark/pull/28715#discussion_r440525221 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala ## @@ -226,18 +226,33 @@ case class GenerateExec( "0"

[GitHub] [spark] maropu commented on a change in pull request #28715: [SPARK-31897][SQL]Enable codegen for GenerateExec

maropu commented on a change in pull request #28715: URL: https://github.com/apache/spark/pull/28715#discussion_r440525322 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/GenerateExec.scala ## @@ -226,18 +226,33 @@ case class GenerateExec( "0"

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644463707 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644463707 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644407813 **[Test build #124076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124076/testReport)** for PR 28708 at commit [`6a77220`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644463033 **[Test build #124076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124076/testReport)** for PR 28708 at commit [`6a77220`](https://github.co

[GitHub] [spark] maropu commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

maropu commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644461527 Btw, could you brush up the PR description for better commit logs? what's the proposal of this PR, what's a behaivor change before/after this PR, brabrabra... I feel the curren o

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

AmplabJenkins removed a comment on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644461275 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

AmplabJenkins commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644461275 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

SparkQA commented on pull request #28833: URL: https://github.com/apache/spark/pull/28833#issuecomment-644460954 **[Test build #124080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124080/testReport)** for PR 28833 at commit [`479901d`](https://github.com

[GitHub] [spark] maropu commented on a change in pull request #28833: [SPARK-20680][SQL] Spark-sql do not support for void column datatype

maropu commented on a change in pull request #28833: URL: https://github.com/apache/spark/pull/28833#discussion_r440519426 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/DataTypeParserSuite.scala ## @@ -61,6 +61,7 @@ class DataTypeParserSuite ex

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644459506 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644459506 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644402021 **[Test build #124075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124075/testReport)** for PR 28708 at commit [`d9123f9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644458813 **[Test build #124075 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124075/testReport)** for PR 28708 at commit [`d9123f9`](https://github.co

[GitHub] [spark] github-actions[bot] commented on pull request #27845: [GRAPHX][MINOR] Shortest paths on weighted edges

github-actions[bot] commented on pull request #27845: URL: https://github.com/apache/spark/pull/27845#issuecomment-644458186 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] github-actions[bot] commented on pull request #22163: [SPARK-25166][CORE]Reduce the number of write operations for shuffle write.

github-actions[bot] commented on pull request #22163: URL: https://github.com/apache/spark/pull/22163#issuecomment-644458199 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue ma

[GitHub] [spark] gatorsmile commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

gatorsmile commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644454930 @holdenk The PR description is not clear. The current PR description does not have the first section in the template "What changes were proposed in this pull request?" . Coul

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644450568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644450563 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

AmplabJenkins commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644450563 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

SparkQA removed a comment on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644259675 **[Test build #124069 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124069/testReport)** for PR 28835 at commit [`7908d62`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28835: [WIP][SPARK-31926][TESTS][FOLLOWUP][test-maven] Cleanup the thread local variable of hive metastore

SparkQA commented on pull request #28835: URL: https://github.com/apache/spark/pull/28835#issuecomment-644450238 **[Test build #124069 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124069/testReport)** for PR 28835 at commit [`7908d62`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

AmplabJenkins removed a comment on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-69106 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

AmplabJenkins commented on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-69106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

SparkQA commented on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-68725 **[Test build #124079 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124079/testReport)** for PR 28784 at commit [`1177e9a`](https://github.com

[GitHub] [spark] maropu commented on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

maropu commented on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-67040 retest this please This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] maropu commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

maropu commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r440501541 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -839,6 +839,17 @@ object SQLConf { .checkValues(HiveCaseS

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-62574 **[Test build #124078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124078/testReport)** for PR 28412 at commit [`3412d29`](https://github.com

[GitHub] [spark] agrawaldevesh commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

agrawaldevesh commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r440500977 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,60 @@ private[spark] class Coarse

[GitHub] [spark] AmplabJenkins commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-60678 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-60678 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-60315 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r440499121 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,186 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-60315 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644266765 **[Test build #124072 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124072/testReport)** for PR 28593 at commit [`12b4239`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644439550 **[Test build #124072 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124072/testReport)** for PR 28593 at commit [`12b4239`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644432286 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644432278 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644429050 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644432241 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28695/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644432278 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644429042 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644430543 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins removed a comment on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-644430538 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644430543 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

AmplabJenkins commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-644430538 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r440490267 ## File path: core/src/main/scala/org/apache/spark/deploy/history/HybridStore.scala ## @@ -0,0 +1,182 @@ +/* + * Licensed to the Apache Software Found

[GitHub] [spark] SparkQA commented on pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

SparkQA commented on pull request #28412: URL: https://github.com/apache/spark/pull/28412#issuecomment-644429902 **[Test build #124077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124077/testReport)** for PR 28412 at commit [`025b13c`](https://github.com

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r440489973 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -1167,6 +1172,17 @@ private[history] class FsHistoryP

[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644245311 **[Test build #124066 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124066/testReport)** for PR 28593 at commit [`bc4b62c`](https://gi

[GitHub] [spark] baohe-zhang commented on a change in pull request #28412: [SPARK-31608][CORE][WEBUI] Add a new type of KVStore to make loading UI faster

baohe-zhang commented on a change in pull request #28412: URL: https://github.com/apache/spark/pull/28412#discussion_r440489736 ## File path: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala ## @@ -128,6 +128,10 @@ private[history] class FsHistoryPro

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644429042 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644429026 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28694/ ---

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-644429193 **[Test build #124066 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124066/testReport)** for PR 28593 at commit [`bc4b62c`](https://github.co

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r440485608 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,60 @@ private[spark] class CoarseGraine

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644423839 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28695/ -

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644422385 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28694/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-644407747 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

AmplabJenkins commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-644407747 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644407813 **[Test build #124076 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124076/testReport)** for PR 28708 at commit [`6a77220`](https://github.com

[GitHub] [spark] SparkQA removed a comment on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA removed a comment on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-644226300 **[Test build #124062 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124062/testReport)** for PR 27690 at commit [`0fbeaf3`](https://gi

[GitHub] [spark] SparkQA commented on pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

SparkQA commented on pull request #27690: URL: https://github.com/apache/spark/pull/27690#issuecomment-644406746 **[Test build #124062 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124062/testReport)** for PR 27690 at commit [`0fbeaf3`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

AmplabJenkins removed a comment on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-644405617 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/124

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

AmplabJenkins removed a comment on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-644405609 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

SparkQA removed a comment on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-644176408 **[Test build #124059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124059/testReport)** for PR 28784 at commit [`1177e9a`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

AmplabJenkins commented on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-644405609 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644382538 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644405353 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28784: [SPARK-31957][SQL][test-maven] Cleanup hive scratch dir for the developer api startWithContext

SparkQA commented on pull request #28784: URL: https://github.com/apache/spark/pull/28784#issuecomment-644405146 **[Test build #124059 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124059/testReport)** for PR 28784 at commit [`1177e9a`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28816: [SPARK-31986][SQL] Fix Julian-Gregorian micros rebasing of overlapping local timestamps

AmplabJenkins removed a comment on pull request #28816: URL: https://github.com/apache/spark/pull/28816#issuecomment-644404197 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28816: [SPARK-31986][SQL] Fix Julian-Gregorian micros rebasing of overlapping local timestamps

AmplabJenkins commented on pull request #28816: URL: https://github.com/apache/spark/pull/28816#issuecomment-644404197 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28816: [SPARK-31986][SQL] Fix Julian-Gregorian micros rebasing of overlapping local timestamps

SparkQA removed a comment on pull request #28816: URL: https://github.com/apache/spark/pull/28816#issuecomment-644226261 **[Test build #124063 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124063/testReport)** for PR 28816 at commit [`06ef8c0`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28816: [SPARK-31986][SQL] Fix Julian-Gregorian micros rebasing of overlapping local timestamps

SparkQA commented on pull request #28816: URL: https://github.com/apache/spark/pull/28816#issuecomment-644403131 **[Test build #124063 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124063/testReport)** for PR 28816 at commit [`06ef8c0`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644402021 **[Test build #124075 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124075/testReport)** for PR 28708 at commit [`d9123f9`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

AmplabJenkins removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644396548 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

AmplabJenkins commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644396548 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

SparkQA removed a comment on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644214042 **[Test build #124061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124061/testReport)** for PR 28824 at commit [`b42d105`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

SparkQA commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-644394976 **[Test build #124061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124061/testReport)** for PR 28824 at commit [`b42d105`](https://github.co

[GitHub] [spark] jiangxb1987 commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

jiangxb1987 commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-644394716 Thanks, merged to master! This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] jiangxb1987 closed pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

jiangxb1987 closed pull request #28641: URL: https://github.com/apache/spark/pull/28641 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] holdenk commented on a change in pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28708: URL: https://github.com/apache/spark/pull/28708#discussion_r440443890 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -0,0 +1,281 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] holdenk commented on a change in pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28708: URL: https://github.com/apache/spark/pull/28708#discussion_r440442024 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -0,0 +1,281 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644378828 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644382525 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

AmplabJenkins removed a comment on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-644381786 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

AmplabJenkins commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-644381786 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

SparkQA removed a comment on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-644263214 **[Test build #124071 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124071/testReport)** for PR 28641 at commit [`681d9e5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

SparkQA commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-644380608 **[Test build #124071 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/124071/testReport)** for PR 28641 at commit [`681d9e5`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins removed a comment on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644378818 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28746: [SPARK-31922][CORE] Fix "RpcEnv already stopped" error when exit spark-shell with local-cluster mode

AmplabJenkins commented on pull request #28746: URL: https://github.com/apache/spark/pull/28746#issuecomment-644378818 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-644376619 Hey @gatorsmile I noticed you marked this as "request changes", can you clarify what changes you are requesting? ---

<    1   2   3   4   5   6   7   8   >