[GitHub] [spark] zhli1142015 edited a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 edited a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-642344689 > Looks fine in general - one thing looks to be missed is, `LevelDBIterator.close()` should also call `db.closeIterator(this);` to make sure tracker can indicate it.

[GitHub] [spark] beliefer commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

beliefer commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643549852 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

SparkQA commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643549980 **[Test build #123953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123953/testReport)** for PR 28641 at commit [`681d9e5`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

AmplabJenkins commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643550115 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

AmplabJenkins removed a comment on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643550115 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

SparkQA commented on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643552545 **[Test build #123954 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123954/testReport)** for PR 28814 at commit [`60d6c51`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

AmplabJenkins removed a comment on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643552728 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

AmplabJenkins commented on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643552728 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] agrawaldevesh commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

agrawaldevesh commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643552975 @holdenk, what are your thoughts on reducing the number of fetch failed exceptions (and thus potential job failures) due to decommissioning: - This PR tries to offload

[GitHub] [spark] agrawaldevesh edited a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

agrawaldevesh edited a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643552975 @holdenk, what are your thoughts on reducing the number of fetch failed exceptions (and thus potential job failures) due to decommissioning: - This PR tries to o

[GitHub] [spark] SparkQA commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

SparkQA commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643553626 **[Test build #123946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123946/testReport)** for PR 28807 at commit [`f8cbbb1`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

SparkQA removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643506277 **[Test build #123946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123946/testReport)** for PR 28807 at commit [`f8cbbb1`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643553739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643553739 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643553994 Yeah I think supporting multiple ways of reducing the number of fetch failures makes sense here. I think migration is certainly a "best-case" scenario and we can't count on in m

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643553741 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] shanyu commented on pull request #27843: [SPARK-31029] Avoid using global execution context in driver main thread for YarnSchedulerBackend

shanyu commented on pull request #27843: URL: https://github.com/apache/spark/pull/27843#issuecomment-643554083 Hi @tgravescs are we good to merge this PR? This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] holdenk commented on a change in pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on a change in pull request #28708: URL: https://github.com/apache/spark/pull/28708#discussion_r439700057 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManagerDecommissioner.scala ## @@ -0,0 +1,265 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

SparkQA commented on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643554228 **[Test build #123955 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123955/testReport)** for PR 28814 at commit [`4f0bc9f`](https://github.com

[GitHub] [spark] holdenk opened a new pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk opened a new pull request #28817: URL: https://github.com/apache/spark/pull/28817 ### What changes were proposed in this pull request? Exit the executor when it has been asked to decommission and there is nothing left for it to do. ### Why are the changes needed?

[GitHub] [spark] AmplabJenkins commented on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

AmplabJenkins commented on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643554396 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] agrawaldevesh commented on a change in pull request #27636: [SPARK-30873][CORE][YARN]Handling Node Decommissioning for Yarn cluster manger in Spark

agrawaldevesh commented on a change in pull request #27636: URL: https://github.com/apache/spark/pull/27636#discussion_r439683319 ## File path: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala ## @@ -436,6 +449,72 @@ private[yarn] class Ya

[GitHub] [spark] holdenk opened a new pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk opened a new pull request #28818: URL: https://github.com/apache/spark/pull/28818 This is WIP since it is on top of SPARK-31197 (which itself is WIP on top off SPARK-20629 ) and should probably have more testing. ### What changes were proposed in this pull request? If

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

AmplabJenkins removed a comment on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643554396 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

SparkQA commented on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643555188 **[Test build #123957 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123957/testReport)** for PR 28817 at commit [`ea8efc7`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643555190 **[Test build #123956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123956/testReport)** for PR 28818 at commit [`2ff94ec`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643555300 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28801: [SPARK-31970][CORE] Make MDC configuration step be consistent between setLocalProperty and log4j.properties

SparkQA commented on pull request #28801: URL: https://github.com/apache/spark/pull/28801#issuecomment-643555328 **[Test build #123947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123947/testReport)** for PR 28801 at commit [`2c77bbb`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

AmplabJenkins commented on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643555289 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643555300 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28801: [SPARK-31970][CORE] Make MDC configuration step be consistent between setLocalProperty and log4j.properties

SparkQA removed a comment on pull request #28801: URL: https://github.com/apache/spark/pull/28801#issuecomment-643520932 **[Test build #123947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123947/testReport)** for PR 28801 at commit [`2c77bbb`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

AmplabJenkins removed a comment on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643555289 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28801: [SPARK-31970][CORE] Make MDC configuration step be consistent between setLocalProperty and log4j.properties

AmplabJenkins removed a comment on pull request #28801: URL: https://github.com/apache/spark/pull/28801#issuecomment-643555627 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28801: [SPARK-31970][CORE] Make MDC configuration step be consistent between setLocalProperty and log4j.properties

AmplabJenkins commented on pull request #28801: URL: https://github.com/apache/spark/pull/28801#issuecomment-643555627 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643555962 **[Test build #123958 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123958/testReport)** for PR 28708 at commit [`0ea927d`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643557363 **[Test build #123945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123945/testReport)** for PR 28123 at commit [`cf5b835`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

SparkQA commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643557325 **[Test build #123959 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123959/testReport)** for PR 28807 at commit [`086a2ba`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643557413 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643502389 **[Test build #123945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123945/testReport)** for PR 28123 at commit [`cf5b835`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643557413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643557577 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643557577 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643557580 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] maropu commented on a change in pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

maropu commented on a change in pull request #28807: URL: https://github.com/apache/spark/pull/28807#discussion_r439702584 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/TableIdentifierParserSuite.scala ## @@ -388,12 +391,24 @@ class TableIdenti

[GitHub] [spark] holdenk commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

holdenk commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643559589 Although I'm a little fuzzy on what you mean by "eager" (if you mean as soon as the migrations are completed then yes) -

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643560584 **[Test build #123960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123960/testReport)** for PR 28708 at commit [`fe34308`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643560550 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28580/ -

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643561728 **[Test build #123961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123961/testReport)** for PR 28593 at commit [`e29409d`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643561834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643561834 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] moomindani commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

moomindani commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r439705159 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -124,11 +153,24 @@ private[hive] trait SaveAsHive

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643562280 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28580/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643562282 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643562282 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

SparkQA commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643563372 **[Test build #123944 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123944/testReport)** for PR 28807 at commit [`17b8f2c`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

SparkQA removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643502351 **[Test build #123944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123944/testReport)** for PR 28807 at commit [`17b8f2c`](https://gi

[GitHub] [spark] TJX2014 opened a new pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates

TJX2014 opened a new pull request #28819: URL: https://github.com/apache/spark/pull/28819 ### What changes were proposed in this pull request? Add judge equal condition as bigger in `org.apache.spark.sql.catalyst.expressions.Sequence.TemporalSequenceImpl#eval` Unit for interval `day`,

[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates

AmplabJenkins commented on pull request #28819: URL: https://github.com/apache/spark/pull/28819#issuecomment-643563705 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins commented on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643563711 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28807: [WIP][SPARK-26905][SQL] Follow the SQL:2016 reserved keywords

AmplabJenkins removed a comment on pull request #28807: URL: https://github.com/apache/spark/pull/28807#issuecomment-643563711 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates

AmplabJenkins removed a comment on pull request #28819: URL: https://github.com/apache/spark/pull/28819#issuecomment-643563705 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates

AmplabJenkins commented on pull request #28819: URL: https://github.com/apache/spark/pull/28819#issuecomment-643563821 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] moomindani commented on a change in pull request #27690: [SPARK-21514][SQL] Added a new option to use non-blobstore storage when writing into blobstore storage

moomindani commented on a change in pull request #27690: URL: https://github.com/apache/spark/pull/27690#discussion_r439706256 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/SaveAsHiveFile.scala ## @@ -97,7 +99,34 @@ private[hive] trait SaveAsHiveFil

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r439706394 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala ## @@ -0,0 +1,112 @@ +/* + * License

[GitHub] [spark] imback82 commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

imback82 commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643564022 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r439706576 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala ## @@ -165,6 +166,7 @@ case class FileSourceScanExec(

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643564457 **[Test build #123962 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123962/testReport)** for PR 28123 at commit [`cf5b835`](https://github.com

[GitHub] [spark] imback82 commented on a change in pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

imback82 commented on a change in pull request #28123: URL: https://github.com/apache/spark/pull/28123#discussion_r439706623 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/bucketing/CoalesceBucketsInSortMergeJoin.scala ## @@ -0,0 +1,112 @@ +/* + * License

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643564557 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643564647 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28582/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643564557 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

SparkQA commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643565763 **[Test build #123952 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123952/testReport)** for PR 28769 at commit [`2b8586b`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

SparkQA removed a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643547384 **[Test build #123952 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123952/testReport)** for PR 28769 at commit [`2b8586b`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

AmplabJenkins removed a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643565959 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

AmplabJenkins commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643565959 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

AmplabJenkins removed a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643565961 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643566829 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643566829 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643566824 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28582/ ---

[GitHub] [spark] agrawaldevesh commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

agrawaldevesh commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643566981 > Although I'm a little fuzzy on what you mean by "eager" (if you mean as soon as the migrations are completed then yes) Thank you for confirming ! By *eager*, I spe

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643568905 **[Test build #123962 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123962/testReport)** for PR 28123 at commit [`cf5b835`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643568935 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643568935 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643564457 **[Test build #123962 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123962/testReport)** for PR 28123 at commit [`cf5b835`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643568937 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] sarutak commented on pull request #28752: [SPARK-31642][FOLLOWUP] Fix Sorting for duration column and make Status column sortable

sarutak commented on pull request #28752: URL: https://github.com/apache/spark/pull/28752#issuecomment-643569254 > LGTM. Do you all have an opinion on back porting to branch-3.0? I think I would. Yeah, it would be better. What do you think @iRakson ? -

[GitHub] [spark] sarutak commented on pull request #28806: [SPARK-31967][UI] Downgrade to vis.js 4.21.0 to fix Jobs UI loading time regression

sarutak commented on pull request #28806: URL: https://github.com/apache/spark/pull/28806#issuecomment-643569783 > This downgraded solution is better. In my local setup, the perf issue can't not be reproduced, and by default the infinite timeline redrawing won't happen unless the "Enable z

[GitHub] [spark] imback82 commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

imback82 commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643570195 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins removed a comment on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643570278 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

AmplabJenkins commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643570278 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28123: [SPARK-31350][SQL] Coalesce bucketed tables for sort merge join if applicable

SparkQA commented on pull request #28123: URL: https://github.com/apache/spark/pull/28123#issuecomment-643570837 **[Test build #123963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123963/testReport)** for PR 28123 at commit [`cf5b835`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643570926 **[Test build #123956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123956/testReport)** for PR 28818 at commit [`2ff94ec`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643555190 **[Test build #123956 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123956/testReport)** for PR 28818 at commit [`2ff94ec`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643571010 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643571010 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643571011 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/123

[GitHub] [spark] karuppayya commented on a change in pull request #28804: [SPARK-31973][SQL] Add ability to disable Sort,Spill in Partial aggregation

karuppayya commented on a change in pull request #28804: URL: https://github.com/apache/spark/pull/28804#discussion_r439710508 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala ## @@ -165,6 +166,26 @@ class WholeStageCodegenSuite

[GitHub] [spark] SparkQA removed a comment on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

SparkQA removed a comment on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643549980 **[Test build #123953 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123953/testReport)** for PR 28641 at commit [`681d9e5`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

SparkQA commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643572136 **[Test build #123953 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123953/testReport)** for PR 28641 at commit [`681d9e5`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #28641: [SPARK-31824][CORE][TESTS] DAGSchedulerSuite: Improve and reuse completeShuffleMapStageSuccessfully

AmplabJenkins commented on pull request #28641: URL: https://github.com/apache/spark/pull/28641#issuecomment-643572368 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

<    2   3   4   5   6   7   8   >