[GitHub] [spark] AmplabJenkins removed a comment on pull request #28823: [SPARK-31983][WEBUI][3.0] Fix sorting for duration column in structured streaming tab

AmplabJenkins removed a comment on pull request #28823: URL: https://github.com/apache/spark/pull/28823#issuecomment-643668646 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA removed a comment on pull request #28823: [SPARK-31983][WEBUI][3.0] Fix sorting for duration column in structured streaming tab

SparkQA removed a comment on pull request #28823: URL: https://github.com/apache/spark/pull/28823#issuecomment-643637929 **[Test build #123978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123978/testReport)** for PR 28823 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28823: [SPARK-31983][WEBUI][3.0] Fix sorting for duration column in structured streaming tab

AmplabJenkins commented on pull request #28823: URL: https://github.com/apache/spark/pull/28823#issuecomment-643668646 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28823: [SPARK-31983][WEBUI][3.0] Fix sorting for duration column in structured streaming tab

SparkQA commented on pull request #28823: URL: https://github.com/apache/spark/pull/28823#issuecomment-643668439 **[Test build #123978 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123978/testReport)** for PR 28823 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

AmplabJenkins commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-643676869 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

AmplabJenkins removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-643676869 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643679317 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643683292 **[Test build #123979 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123979/testReport)** for PR 28593 at commit

[GitHub] [spark] SparkQA commented on pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

SparkQA commented on pull request #28824: URL: https://github.com/apache/spark/pull/28824#issuecomment-643665648 **[Test build #123982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123982/testReport)** for PR 28824 at commit

[GitHub] [spark] MaxGekk opened a new pull request #28824: [SPARK-31984][SQL] Make micros rebasing functions via local timestamps pure

MaxGekk opened a new pull request #28824: URL: https://github.com/apache/spark/pull/28824 ### What changes were proposed in this pull request? 1. Set the given time zone as the first parameter of `RebaseDateTime`.`rebaseJulianToGregorianMicros()` and `rebaseGregorianToJulianMicros()`

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643666176 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28604/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643669064 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643669057 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/28604/

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643669064 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643669066 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA removed a comment on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

SparkQA removed a comment on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-643659107 **[Test build #123980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123980/testReport)** for PR 28710 at commit

[GitHub] [spark] SparkQA commented on pull request #28710: [SPARK-31893][ML] Add a generic ClassificationSummary trait

SparkQA commented on pull request #28710: URL: https://github.com/apache/spark/pull/28710#issuecomment-643676654 **[Test build #123980 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123980/testReport)** for PR 28710 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

AmplabJenkins commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643679317 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA removed a comment on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643660661 **[Test build #123981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123981/testReport)** for PR 28708 at commit

[GitHub] [spark] SparkQA commented on pull request #28708: [SPARK-20629][CORE][K8S] Copy shuffle data when nodes are being shutdown

SparkQA commented on pull request #28708: URL: https://github.com/apache/spark/pull/28708#issuecomment-643679103 **[Test build #123981 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123981/testReport)** for PR 28708 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

AmplabJenkins commented on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643683462 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28593: [SPARK-31710][SQL] Fail casting numeric to timestamp by default

SparkQA removed a comment on pull request #28593: URL: https://github.com/apache/spark/pull/28593#issuecomment-643651654 **[Test build #123979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123979/testReport)** for PR 28593 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

HeartSaVioR commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-643705743 I can even tolerate the fact maxFileAge is originated from path's latest timestamp. If we don't believe the node's wall time (I suspect other logic works well in such case

[GitHub] [spark] zhli1142015 commented on a change in pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on a change in pull request #28769: URL: https://github.com/apache/spark/pull/28769#discussion_r439787376 ## File path: common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDB.java ## @@ -229,6 +241,10 @@ public long count(Class type, String

[GitHub] [spark] AmplabJenkins commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

AmplabJenkins commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643711604 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

AmplabJenkins removed a comment on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643711604 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643712139 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643712138 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] HyukjinKwon commented on pull request #28819: [SPARK-31980][SQL]Function sequence() fails if start and end of range are equal dates

HyukjinKwon commented on pull request #28819: URL: https://github.com/apache/spark/pull/28819#issuecomment-643722914 I think it matches the behaviour with Presto's (see https://github.com/apache/spark/pull/21155). Shall we check how it works in Presto?

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781710 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,60 @@ private[spark] class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

HeartSaVioR commented on a change in pull request #28769: URL: https://github.com/apache/spark/pull/28769#discussion_r439782275 ## File path: common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDB.java ## @@ -229,6 +241,10 @@ public long count(Class type, String

[GitHub] [spark] zhli1142015 commented on a change in pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on a change in pull request #28769: URL: https://github.com/apache/spark/pull/28769#discussion_r439787328 ## File path: common/kvstore/src/test/java/org/apache/spark/util/kvstore/LevelDBSuite.java ## @@ -276,6 +277,41 @@ public void

[GitHub] [spark] zhli1142015 commented on a change in pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on a change in pull request #28769: URL: https://github.com/apache/spark/pull/28769#discussion_r439787366 ## File path: common/kvstore/src/test/java/org/apache/spark/util/kvstore/LevelDBSuite.java ## @@ -276,6 +277,41 @@ public void

[GitHub] [spark] SparkQA commented on pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

SparkQA commented on pull request #28769: URL: https://github.com/apache/spark/pull/28769#issuecomment-643711520 **[Test build #123994 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123994/testReport)** for PR 28769 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643714697 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

SparkQA commented on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643714577 **[Test build #123995 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123995/testReport)** for PR 28818 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

SparkQA removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643706423 **[Test build #123987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123987/testReport)** for PR 27694 at commit

[GitHub] [spark] SparkQA commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

SparkQA commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643721948 **[Test build #123987 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123987/testReport)** for PR 27694 at commit

[GitHub] [spark] github-actions[bot] commented on pull request #27053: [WIP][SPARK-27495][Core][YARN][k8s] Stage Level Scheduling code for reference

github-actions[bot] commented on pull request #27053: URL: https://github.com/apache/spark/pull/27053#issuecomment-643699205 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] github-actions[bot] commented on pull request #27375: [SPARK-30664][Web UI] Add optional metrics to all-stages page

github-actions[bot] commented on pull request #27375: URL: https://github.com/apache/spark/pull/27375#issuecomment-643699201 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781237 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,65 @@ private[spark] class

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781276 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,65 @@ private[spark] class

[GitHub] [spark] AmplabJenkins commented on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

AmplabJenkins commented on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643703413 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

AmplabJenkins removed a comment on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643703413 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] HeartSaVioR commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

HeartSaVioR commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-643706110 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HeartSaVioR commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

HeartSaVioR commented on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643706118 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HeartSaVioR commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

HeartSaVioR commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-643706117 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun closed pull request #28822: [SPARK-31644][BUILD][FOLLOWUP] Make Spark's guava version configurable from the command line for sbt

dongjoon-hyun closed pull request #28822: URL: https://github.com/apache/spark/pull/28822 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

SparkQA removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643706415 **[Test build #123988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123988/testReport)** for PR 27649 at commit

[GitHub] [spark] SparkQA commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

SparkQA commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643712100 **[Test build #123988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123988/testReport)** for PR 27649 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643712138 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

AmplabJenkins removed a comment on pull request #28818: URL: https://github.com/apache/spark/pull/28818#issuecomment-643714697 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp

AmplabJenkins removed a comment on pull request #28821: URL: https://github.com/apache/spark/pull/28821#issuecomment-643600098 Can one of the admins verify this patch? This is an automated message from the Apache Git

[GitHub] [spark] HyukjinKwon commented on pull request #28821: [SPARK-31981][SQL] Keep TimestampType when taking an average of a Timestamp

HyukjinKwon commented on pull request #28821: URL: https://github.com/apache/spark/pull/28821#issuecomment-643719783 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781338 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedClusterMessage.scala ## @@ -52,6 +52,8 @@ private[spark] object

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781436 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -2039,8 +2047,11 @@ private[spark] class BlockManager( * Class to

[GitHub] [spark] SparkQA commented on pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

SparkQA commented on pull request #28817: URL: https://github.com/apache/spark/pull/28817#issuecomment-643703342 **[Test build #123983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123983/testReport)** for PR 28817 at commit

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781961 ## File path: core/src/main/scala/org/apache/spark/storage/BlockManager.scala ## @@ -1887,7 +1891,7 @@ private[spark] class BlockManager( * but

[GitHub] [spark] AmplabJenkins commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-643706484 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643706496 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files

AmplabJenkins commented on pull request #27620: URL: https://github.com/apache/spark/pull/27620#issuecomment-643706511 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

AmplabJenkins commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-643706522 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643706496 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] wangyum commented on pull request #28032: [SPARK-31264][SQL] Repartition by dynamic partition columns before insert partition table

wangyum commented on pull request #28032: URL: https://github.com/apache/spark/pull/28032#issuecomment-643706474 > @wangyum Question, if we have a repartition hint on p1 and p2 in the SELECT query would it have similar effect ? Yes. It have similar effect.

[GitHub] [spark] AmplabJenkins commented on pull request #24173: [SPARK-27237][SS] Introduce State schema validation among query restart

AmplabJenkins commented on pull request #24173: URL: https://github.com/apache/spark/pull/24173#issuecomment-643706534 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files

AmplabJenkins removed a comment on pull request #27620: URL: https://github.com/apache/spark/pull/27620#issuecomment-643706511 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins removed a comment on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643706508 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins removed a comment on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-643706531 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

AmplabJenkins commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-643706528 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #24173: [SPARK-27237][SS] Introduce State schema validation among query restart

AmplabJenkins removed a comment on pull request #24173: URL: https://github.com/apache/spark/pull/24173#issuecomment-643706534 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

AmplabJenkins removed a comment on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-643706522 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

AmplabJenkins commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643706508 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

AmplabJenkins removed a comment on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643706485 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

AmplabJenkins commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-643706501 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

AmplabJenkins commented on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643706485 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

AmplabJenkins removed a comment on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-643706484 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

AmplabJenkins removed a comment on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-643706501 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

AmplabJenkins commented on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643721071 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

SparkQA removed a comment on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643706416 **[Test build #123984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit

[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

SparkQA commented on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643720854 **[Test build #123984 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

AmplabJenkins removed a comment on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643721071 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dongjoon-hyun commented on pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

dongjoon-hyun commented on pull request #28814: URL: https://github.com/apache/spark/pull/28814#issuecomment-643720843 Hi, @TJX2014 . What is your JIRA id? This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643722166 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

SparkQA commented on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643706423 **[Test build #123987 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123987/testReport)** for PR 27694 at commit

[GitHub] [spark] SparkQA commented on pull request #27649: [SPARK-30900][SS] FileStreamSource: Avoid reading compact metadata log twice if the query restarts from compact batch

SparkQA commented on pull request #27649: URL: https://github.com/apache/spark/pull/27649#issuecomment-643706415 **[Test build #123988 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123988/testReport)** for PR 27649 at commit

[GitHub] [spark] SparkQA commented on pull request #28422: [SPARK-17604][SS] FileStreamSource: provide a new option to have retention on input files

SparkQA commented on pull request #28422: URL: https://github.com/apache/spark/pull/28422#issuecomment-643706409 **[Test build #123985 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123985/testReport)** for PR 28422 at commit

[GitHub] [spark] SparkQA commented on pull request #28607: [SPARK-24634][SS] Add a new metric regarding number of inputs later than watermark plus allowed delay

SparkQA commented on pull request #28607: URL: https://github.com/apache/spark/pull/28607#issuecomment-643706416 **[Test build #123984 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123984/testReport)** for PR 28607 at commit

[GitHub] [spark] SparkQA commented on pull request #27620: [SPARK-30866][SS] FileStreamSource: Cache fetched list of files beyond maxFilesPerTrigger as unread files

SparkQA commented on pull request #27620: URL: https://github.com/apache/spark/pull/27620#issuecomment-643706447 **[Test build #123989 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123989/testReport)** for PR 27620 at commit

[GitHub] [spark] SparkQA commented on pull request #24173: [SPARK-27237][SS] Introduce State schema validation among query restart

SparkQA commented on pull request #24173: URL: https://github.com/apache/spark/pull/24173#issuecomment-643706432 **[Test build #123993 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123993/testReport)** for PR 24173 at commit

[GitHub] [spark] SparkQA commented on pull request #26935: [SPARK-30294][SS] Explicitly defines read-only StateStore and optimize for HDFSBackedStateStore

SparkQA commented on pull request #26935: URL: https://github.com/apache/spark/pull/26935#issuecomment-643706446 **[Test build #123991 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123991/testReport)** for PR 26935 at commit

[GitHub] [spark] SparkQA commented on pull request #25965: [SPARK-26425][SS] Add more constraint checks to avoid checkpoint corruption

SparkQA commented on pull request #25965: URL: https://github.com/apache/spark/pull/25965#issuecomment-643706439 **[Test build #123992 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123992/testReport)** for PR 25965 at commit

[GitHub] [spark] SparkQA commented on pull request #27333: [SPARK-29438][SS][FOLLOWUP] Add regression tests for Streaming Aggregation and flatMapGroupsWithState

SparkQA commented on pull request #27333: URL: https://github.com/apache/spark/pull/27333#issuecomment-643706441 **[Test build #123990 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123990/testReport)** for PR 27333 at commit

[GitHub] [spark] SparkQA commented on pull request #28363: [SPARK-27188][SS] FileStreamSink: provide a new option to have retention on output files

SparkQA commented on pull request #28363: URL: https://github.com/apache/spark/pull/28363#issuecomment-643706412 **[Test build #123986 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/123986/testReport)** for PR 28363 at commit

[GitHub] [spark] dongjoon-hyun closed pull request #28814: [SPARK-31968][SQL]Duplicate partition columns check when writing data

dongjoon-hyun closed pull request #28814: URL: https://github.com/apache/spark/pull/28814 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #27694: [SPARK-30946][SS] Serde entry via DataInputStream/DataOutputStream with LZ4 compression on FileStream(Source/Sink)Log

AmplabJenkins removed a comment on pull request #27694: URL: https://github.com/apache/spark/pull/27694#issuecomment-643722166 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] sarutak commented on pull request #28820: [SPARK-31632][CORE][WEBUI][FOLLOWUP] Enrich the exception message when application summary is unavailable

sarutak commented on pull request #28820: URL: https://github.com/apache/spark/pull/28820#issuecomment-643722372 @HyukjinKwon Sorry I didn't think it's not necessary to file because this is just a small followup. I'll take care. Thanks.

[GitHub] [spark] HyukjinKwon commented on pull request #28391: [SPARK-31593][SS] Remove unnecessary streaming query progress update

HyukjinKwon commented on pull request #28391: URL: https://github.com/apache/spark/pull/28391#issuecomment-643722422 Merged to master and branch-3.0. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] holdenk commented on a change in pull request #28817: [WIP][SPARK-31197][CORE] Exit the executor once all tasks and migrations are finished built on top of on top of spark20629

holdenk commented on a change in pull request #28817: URL: https://github.com/apache/spark/pull/28817#discussion_r439781822 ## File path: core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala ## @@ -258,26 +262,65 @@ private[spark] class

[GitHub] [spark] holdenk commented on a change in pull request #28818: [WIP][SPARK-31198][CORE] Use graceful decommissioning as part of dynamic scaling

holdenk commented on a change in pull request #28818: URL: https://github.com/apache/spark/pull/28818#discussion_r439785650 ## File path: core/src/main/scala/org/apache/spark/scheduler/dynalloc/ExecutorMonitor.scala ## @@ -333,11 +335,19 @@ private[spark] class

[GitHub] [spark] zhli1142015 commented on a change in pull request #28769: [SPARK-31929][WEBUI] Close leveldbiterator when leveldb.close

zhli1142015 commented on a change in pull request #28769: URL: https://github.com/apache/spark/pull/28769#discussion_r439787481 ## File path: common/kvstore/src/main/java/org/apache/spark/util/kvstore/LevelDB.java ## @@ -238,6 +254,14 @@ public void close() throws IOException

<    1   2   3   4   >