[GitHub] [spark] iRakson commented on a change in pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

iRakson commented on a change in pull request #28485: URL: https://github.com/apache/spark/pull/28485#discussion_r422815037 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/ui/UISeleniumSuite.scala ## @@ -86,34 +86,36 @@ class UISeleniumSuite extends SparkF

[GitHub] [spark] uncleGen commented on a change in pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

uncleGen commented on a change in pull request #28485: URL: https://github.com/apache/spark/pull/28485#discussion_r422801194 ## File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/ui/StreamingQueryPage.scala ## @@ -35,11 +40,218 @@ private[ui] class StreamingQuer

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626498246 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626498246 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

SparkQA removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626432135 **[Test build #122485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122485/testReport)** for PR 28485 at commit [`df61b7c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

SparkQA commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626497339 **[Test build #122485 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122485/testReport)** for PR 28485 at commit [`df61b7c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28477: [SPARK-31405][SQL] Fail by default when reading/writing legacy datetime values from/to Parquet/Avro files

SparkQA commented on pull request #28477: URL: https://github.com/apache/spark/pull/28477#issuecomment-626495706 **[Test build #122489 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122489/testReport)** for PR 28477 at commit [`fc31eea`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28477: [SPARK-31405][SQL] Fail by default when reading/writing legacy datetime values from/to Parquet/Avro files

AmplabJenkins removed a comment on pull request #28477: URL: https://github.com/apache/spark/pull/28477#issuecomment-626493276 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28477: [SPARK-31405][SQL] Fail by default when reading/writing legacy datetime values from/to Parquet/Avro files

AmplabJenkins commented on pull request #28477: URL: https://github.com/apache/spark/pull/28477#issuecomment-626493276 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] igreenfield commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

igreenfield commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r422802737 ## File path: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ## @@ -157,23 +259,23 @@ private[spark] object ThreadUtils { */ def n

[GitHub] [spark] MaxGekk commented on a change in pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

MaxGekk commented on a change in pull request #28481: URL: https://github.com/apache/spark/pull/28481#discussion_r422800558 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala ## @@ -162,52 +162,66 @@ object RandomDataGenerator { })

[GitHub] [spark] MaxGekk commented on a change in pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

MaxGekk commented on a change in pull request #28481: URL: https://github.com/apache/spark/pull/28481#discussion_r422799964 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala ## @@ -162,52 +162,66 @@ object RandomDataGenerator { })

[GitHub] [spark] Gschiavon removed a comment on pull request #27578: [SPARK-30828][SQL] Improving insertInto behaviour

Gschiavon removed a comment on pull request #27578: URL: https://github.com/apache/spark/pull/27578#issuecomment-593260363 ping @dongjoon-hyun Hi! what do you think about this? This is an automated message from the Apache

[GitHub] [spark] Gschiavon removed a comment on pull request #27578: [SPARK-30828][SQL] Improving insertInto behaviour

Gschiavon removed a comment on pull request #27578: URL: https://github.com/apache/spark/pull/27578#issuecomment-626487699 @AmplabJenkins This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] Gschiavon commented on pull request #27578: [SPARK-30828][SQL] Improving insertInto behaviour

Gschiavon commented on pull request #27578: URL: https://github.com/apache/spark/pull/27578#issuecomment-626487699 @AmplabJenkins This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] maropu commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

maropu commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626485974 The change looks useful to me. cc: @MaxGekk This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] Ngone51 commented on a change in pull request #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF

Ngone51 commented on a change in pull request #27937: URL: https://github.com/apache/spark/pull/27937#discussion_r422793783 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala ## @@ -93,7 +93,7 @@ sealed abstract class UserDefinedFu

[GitHub] [spark] cloud-fan commented on a change in pull request #26624: [SPARK-8981][CORE][test-hadoop3.2][test-java11] Add MDC support in Executor

cloud-fan commented on a change in pull request #26624: URL: https://github.com/apache/spark/pull/26624#discussion_r422791868 ## File path: core/src/main/scala/org/apache/spark/util/ThreadUtils.scala ## @@ -157,23 +259,23 @@ private[spark] object ThreadUtils { */ def new

[GitHub] [spark] dongjoon-hyun commented on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

dongjoon-hyun commented on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626478958 Thank you so much, @dbtsai . Merged to master/3.0. This is an automated message from the Apache Git Servic

[GitHub] [spark] beliefer commented on pull request #28164: [SPARK-31393][SQL] Show the correct alias in schema for expression

beliefer commented on pull request #28164: URL: https://github.com/apache/spark/pull/28164#issuecomment-626477321 cc @gatorsmile This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] dbtsai commented on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

dbtsai commented on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626475204 LGTM. Thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] dilipbiswal commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

dilipbiswal commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626473070 cc @gatorsmile @maropu This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

cloud-fan commented on a change in pull request #28481: URL: https://github.com/apache/spark/pull/28481#discussion_r422784030 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/RandomDataGenerator.scala ## @@ -162,52 +162,66 @@ object RandomDataGenerator { }

[GitHub] [spark] cloud-fan commented on pull request #28489: [SPARK-31672][SQL] Fix loading of timestamps before 1582-10-15 from dictionary encoded Parquet columns

cloud-fan commented on pull request #28489: URL: https://github.com/apache/spark/pull/28489#issuecomment-626469503 LGTM, merging to master/3.0! This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [spark] cloud-fan commented on a change in pull request #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF

cloud-fan commented on a change in pull request #27937: URL: https://github.com/apache/spark/pull/27937#discussion_r422778641 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala ## @@ -93,7 +93,7 @@ sealed abstract class UserDefined

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

AmplabJenkins removed a comment on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626465447 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

AmplabJenkins commented on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626465447 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

SparkQA commented on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626465224 **[Test build #122488 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122488/testReport)** for PR 28481 at commit [`a78e18b`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #28486: [SPARK-31669][SQL][TESTS] Fix RowEncoderSuite failures on non-existing dates/timestamps

cloud-fan commented on pull request #28486: URL: https://github.com/apache/spark/pull/28486#issuecomment-626464599 Can we generate `LocalDate`/`Instant`? Spark supports these 2 types when creating DataFrame/Dataset. This is

[GitHub] [spark] MaxGekk commented on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

MaxGekk commented on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626464195 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

AmplabJenkins removed a comment on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626463755 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

AmplabJenkins commented on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626463755 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

dongjoon-hyun commented on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626463597 Could you review this PR when you have some time, @dbtsai ? This is an automated message from the Apache G

[GitHub] [spark] SparkQA commented on pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

SparkQA commented on pull request #28495: URL: https://github.com/apache/spark/pull/28495#issuecomment-626463445 **[Test build #122487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122487/testReport)** for PR 28495 at commit [`bad1068`](https://github.com

[GitHub] [spark] dongjoon-hyun opened a new pull request #28495: [SPARK-31674][CORE][DOCS] Make Prometheus metric endpoints experimental

dongjoon-hyun opened a new pull request #28495: URL: https://github.com/apache/spark/pull/28495 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was t

[GitHub] [spark] huaxingao commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

huaxingao commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-626460022 @gatorsmile @dilipbiswal Anything else you want to add in 3.0? This is an automated message from the Apache Gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

AmplabJenkins removed a comment on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626453008 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

AmplabJenkins commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626453008 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

SparkQA removed a comment on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626438326 **[Test build #122486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122486/testReport)** for PR 28413 at commit [`63167fd`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

SparkQA commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626452744 **[Test build #122486 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122486/testReport)** for PR 28413 at commit [`63167fd`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

AmplabJenkins removed a comment on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626450849 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

AmplabJenkins commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626450849 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

SparkQA removed a comment on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626403485 **[Test build #122484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122484/testReport)** for PR 28493 at commit [`62dbc83`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

SparkQA commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626450430 **[Test build #122484 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122484/testReport)** for PR 28493 at commit [`62dbc83`](https://github.co

[GitHub] [spark] HeartSaVioR commented on pull request #28336: [SPARK-31559][YARN] Re-obtain tokens at the startup of AM for yarn cluster mode if principal and keytab are available

HeartSaVioR commented on pull request #28336: URL: https://github.com/apache/spark/pull/28336#issuecomment-626446309 Appreciate another round of reviews. Thanks in advance! This is an automated message from the Apache Git Ser

[GitHub] [spark] koertkuipers commented on a change in pull request #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF

koertkuipers commented on a change in pull request #27937: URL: https://github.com/apache/spark/pull/27937#discussion_r422699979 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala ## @@ -93,7 +93,7 @@ sealed abstract class UserDefi

[GitHub] [spark] Ngone51 commented on a change in pull request #28444: [SPARK-31632][CORE][WEBUI] Make the ApplicationInfo always available when accessed

Ngone51 commented on a change in pull request #28444: URL: https://github.com/apache/spark/pull/28444#discussion_r422750454 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -35,8 +36,24 @@ private[spark] class AppStatusStore( val store:

[GitHub] [spark] Ngone51 commented on a change in pull request #28258: [SPARK-31486] [CORE] spark.submit.waitAppCompletion flag to control spark-submit exit in Standalone Cluster Mode

Ngone51 commented on a change in pull request #28258: URL: https://github.com/apache/spark/pull/28258#discussion_r422744560 ## File path: core/src/main/scala/org/apache/spark/deploy/Client.scala ## @@ -176,6 +190,25 @@ private class ClientEndpoint( } else if (!Utils.resp

[GitHub] [spark] sarutak commented on a change in pull request #28448: [SPARK-31638][WEBUI]Clean Pagination code for all webUI pages

sarutak commented on a change in pull request #28448: URL: https://github.com/apache/spark/pull/28448#discussion_r422748282 ## File path: core/src/main/scala/org/apache/spark/ui/jobs/AllJobsPage.scala ## @@ -523,13 +517,11 @@ private[ui] class JobPagedTable( store, da

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

AmplabJenkins removed a comment on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626438622 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

AmplabJenkins commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626438622 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28413: [SPARK-31610][SPARK-31668][ML] Address hashingTF saving&loading bug and expose hashFunc property in HashingTF

SparkQA commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626438326 **[Test build #122486 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122486/testReport)** for PR 28413 at commit [`63167fd`](https://github.com

[GitHub] [spark] WeichenXu123 commented on pull request #28413: [SPARK-31610][ML] Expose hashFunc property in HashingTF

WeichenXu123 commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626436050 @huaxingao I will integrate the change into my PR. :) This is an automated message from the Apache Git Serv

[GitHub] [spark] uncleGen edited a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

uncleGen edited a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626431760 Seems like an unrelated and flaky failure and will fixed in #28391 This is an automated message from th

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626432365 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626432365 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

SparkQA commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626432135 **[Test build #122485 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122485/testReport)** for PR 28485 at commit [`df61b7c`](https://github.com

[GitHub] [spark] uncleGen commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

uncleGen commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626431951 Jenkins retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] uncleGen edited a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

uncleGen edited a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626431760 Seems like an unrelated failure and will fixed in #28391 This is an automated message from the Apache G

[GitHub] [spark] uncleGen commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

uncleGen commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626431760 Retest this please. Seems like an unrelated failure and will fixed in #28391 This is an automated message from

[GitHub] [spark] maropu commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

maropu commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-626421386 All the document works for 3.0 have been done? https://issues.apache.org/jira/browse/SPARK-28588 This is an auto

[GitHub] [spark] maropu commented on pull request #28368: [SPARK-31575][SQL] Synchronise global JVM security configuration modification

maropu commented on pull request #28368: URL: https://github.com/apache/spark/pull/28368#issuecomment-626420310 This feature (Kerberos support for JDBC connections) is only for 3.1, so I think this issue doesn't happen in 3.0/2.4. --

[GitHub] [spark] huaxingao commented on pull request #28413: [SPARK-31610][ML] Expose hashFunc property in HashingTF

huaxingao commented on pull request #28413: URL: https://github.com/apache/spark/pull/28413#issuecomment-626408391 @WeichenXu123 Do you want to integrate the change into your PR? Or you want me to submit a PR after yours is merged? I am OK either way.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

AmplabJenkins removed a comment on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626407379 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

AmplabJenkins removed a comment on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626407376 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

SparkQA removed a comment on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626388490 **[Test build #122483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122483/testReport)** for PR 28482 at commit [`fa6caf0`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

AmplabJenkins commented on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626407376 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

SparkQA commented on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626407268 **[Test build #122483 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122483/testReport)** for PR 28482 at commit [`fa6caf0`](https://github.co

[GitHub] [spark] huaxingao commented on pull request #28451: [SPARK-31636][SQL][DOCS] Remove HTML syntax in SQL reference

huaxingao commented on pull request #28451: URL: https://github.com/apache/spark/pull/28451#issuecomment-626406578 Thanks all! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

AmplabJenkins removed a comment on pull request #28488: URL: https://github.com/apache/spark/pull/28488#issuecomment-626404988 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

AmplabJenkins removed a comment on pull request #28488: URL: https://github.com/apache/spark/pull/28488#issuecomment-626404987 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

SparkQA removed a comment on pull request #28488: URL: https://github.com/apache/spark/pull/28488#issuecomment-626386553 **[Test build #122482 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122482/testReport)** for PR 28488 at commit [`d0957cb`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

AmplabJenkins commented on pull request #28488: URL: https://github.com/apache/spark/pull/28488#issuecomment-626404987 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

SparkQA commented on pull request #28488: URL: https://github.com/apache/spark/pull/28488#issuecomment-626404817 **[Test build #122482 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122482/testReport)** for PR 28488 at commit [`d0957cb`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28494: SPARK-31456 Fix shutdown hook priority edge cases

AmplabJenkins removed a comment on pull request #28494: URL: https://github.com/apache/spark/pull/28494#issuecomment-626404335 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #28494: SPARK-31456 Fix shutdown hook priority edge cases

AmplabJenkins commented on pull request #28494: URL: https://github.com/apache/spark/pull/28494#issuecomment-626404498 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins commented on pull request #28494: SPARK-31456 Fix shutdown hook priority edge cases

AmplabJenkins commented on pull request #28494: URL: https://github.com/apache/spark/pull/28494#issuecomment-626404335 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

AmplabJenkins removed a comment on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626403637 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] oleg-smith opened a new pull request #28494: SPARK-31456 Fix shutdown hook priority edge cases

oleg-smith opened a new pull request #28494: URL: https://github.com/apache/spark/pull/28494 ### What changes were proposed in this pull request? Fix application order for shutdown hooks for the priorities of Int.MaxValue, Int.MinValue ### Why are the changes neede

[GitHub] [spark] AmplabJenkins commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

AmplabJenkins commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626403637 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

SparkQA commented on pull request #28493: URL: https://github.com/apache/spark/pull/28493#issuecomment-626403485 **[Test build #122484 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122484/testReport)** for PR 28493 at commit [`62dbc83`](https://github.com

[GitHub] [spark] dilipbiswal opened a new pull request #28493: [SPARK-31673][SQL] QueryExection.debug.toFile() to take an addtional explain mode param.

dilipbiswal opened a new pull request #28493: URL: https://github.com/apache/spark/pull/28493 ### What changes were proposed in this pull request? Currently QueryExecution.debug.toFile dumps the query plan information in a fixed format. This PR adds an additional explain mode parameter t

[GitHub] [spark] srowen commented on a change in pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

srowen commented on a change in pull request #28488: URL: https://github.com/apache/spark/pull/28488#discussion_r422711668 ## File path: core/src/main/scala/org/apache/spark/rdd/RDD.scala ## @@ -1031,20 +1033,24 @@ abstract class RDD[T: ClassTag]( Array.concat(results: _*)

[GitHub] [spark] felixcheung commented on pull request #28386: [SPARK-26199][SPARK-31517][R] fix strategy for handling ... names in mutate

felixcheung commented on pull request #28386: URL: https://github.com/apache/spark/pull/28386#issuecomment-626400503 Is this a problem with only the new R 4.0.0 release? Maybe Spark doesn’t support it for now, if there isn’t a clean (or software license compatible) way to do this?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

AmplabJenkins removed a comment on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626400072 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] SparkQA removed a comment on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

SparkQA removed a comment on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626379984 **[Test build #122481 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122481/testReport)** for PR 28481 at commit [`a78e18b`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

AmplabJenkins removed a comment on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626400070 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins commented on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

AmplabJenkins commented on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626400070 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28481: [SPARK-31665][SQL][TESTS] Check parquet dictionary encoding of random dates/timestamps

SparkQA commented on pull request #28481: URL: https://github.com/apache/spark/pull/28481#issuecomment-626400020 **[Test build #122481 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122481/testReport)** for PR 28481 at commit [`a78e18b`](https://github.co

[GitHub] [spark] koertkuipers commented on a change in pull request #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF

koertkuipers commented on a change in pull request #27937: URL: https://github.com/apache/spark/pull/27937#discussion_r422699979 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala ## @@ -93,7 +93,7 @@ sealed abstract class UserDefi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626394270 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/122

[GitHub] [spark] AmplabJenkins commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626394264 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

AmplabJenkins removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626394264 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

SparkQA removed a comment on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626366090 **[Test build #122479 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122479/testReport)** for PR 28485 at commit [`df61b7c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #28485: [SPARK-31642] Add Pagination Support for Structured Streaming Page

SparkQA commented on pull request #28485: URL: https://github.com/apache/spark/pull/28485#issuecomment-626394163 **[Test build #122479 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122479/testReport)** for PR 28485 at commit [`df61b7c`](https://github.co

[GitHub] [spark] koertkuipers commented on a change in pull request #27937: [SPARK-30127][SQL] Support case class parameter for typed Scala UDF

koertkuipers commented on a change in pull request #27937: URL: https://github.com/apache/spark/pull/27937#discussion_r422699979 ## File path: sql/core/src/main/scala/org/apache/spark/sql/expressions/UserDefinedFunction.scala ## @@ -93,7 +93,7 @@ sealed abstract class UserDefi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

AmplabJenkins removed a comment on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626388652 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

AmplabJenkins commented on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626388652 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #28482: [DONOTMERGE][DEBUG] Debug the test issues in SPARK-20732

SparkQA commented on pull request #28482: URL: https://github.com/apache/spark/pull/28482#issuecomment-626388490 **[Test build #122483 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/122483/testReport)** for PR 28482 at commit [`fa6caf0`](https://github.com

[GitHub] [spark] oleg-smith commented on a change in pull request #28488: [SPARK-29083][CORE] Prefetch elements in rdd.toLocalIterator

oleg-smith commented on a change in pull request #28488: URL: https://github.com/apache/spark/pull/28488#discussion_r422698223 ## File path: core/src/main/scala/org/apache/spark/rdd/RDD.scala ## @@ -1031,20 +1033,24 @@ abstract class RDD[T: ClassTag]( Array.concat(results:

  1   2   3   >